IDA 


December  1996 

Approved  for  public  release; 
distribution  unlimited. 

IDA  Document  D-1941 

Log:  H  96-004086 


NSTITUTE  FOR  DEFENSE  ANALYSES 


Distance  Learning  and  the 
Reserve  Components 


J.  Metzko 
G.  A.  Redding 
J.  D.  Fletcher 


DTIC  QUALTI'Y  mSPSGTSD  4 

19970213  036 


/■ 


This  work  was  conducted  under  contract  DASW01  94  C  0054,  Task 
T-L2-1374,  for  the  Office  of  the  Under  Secretary  of  Defense  (Personnel 
and  Readiness).  The  publication  of  this  IDA  document  does  not  indicate 
endorsement  by  the  Department  of  Defense,  nor  should  the  contents  be 
construed  as  reflecting  the  official  position  of  that  Agency. 

©  1996, 1997  Institute  for  Defense  Analyses,  1801 N.  Beauregard  Street, 
Alexandria,  Virginia  22311-1772  •  (703)845-2000. 

This  material  may  be  reproduced  by  or  for  the  U.S.  Government  pursuant 
to  the  copyright  license  under  the  clause  at  DFARS  252.227-7013 
(10/88). 


INSTITUTE  FOR  DEFENSE  ANALYSES 


# 


IDA  Document  D-1941 


Distance  Learning  and  the 
Reserve  Components 


J.  Metzko 
G.  A.  Redding 
J.  D.  Fletcher 


# 


PREFACE 


This  report  documents  an  analytical  effort  performed  under  a  task  entitled  ‘Total 
Force  Distance  Learning  Assessment.”  An  earlier  version  of  the  report  was  used  by  the 
staffs  of  the  Office  of  the  Deputy  Under  Secretary  of  Defense/Readiness  (ODUSD/R)  and 
the  Office  of  the  Assistant  Secretary  of  Defense/Reserve  Affairs  (OASD/RA)  to  identify  and 
evaluate  Service  plans  to  adopt  Distance  Learning  (DL)  technologies.  While  the  analytical 
findings  have  not  changed,  substantially  more  evidence — discussion  and  citations  of 
numerous  studies — of  the  effectiveness  and  cost  effectiveness  of  DL  technologies  has  been 
added  in  this  expanded  report,  the  data  of  which  were  current  through  FY95. 

Two  staff  members  of  the  Office  of  the  Secretary  of  Defense  desCTve  grateful 
recognition  for  their  suggestions  and  encouragement  throughout  the  development  of  this 
report:  Mr.  Gary  Boycan,  ODUSD/R,  and  Commander  Lonie  Rezendes,  USNR,  formerly 
in  OASD/RA  before  reassignment  to  Headquarters,  Naval  Sea  Systems  Command. 


CONTENTS 


Executive  Summary . S-1 

I.  BsrmoDucnoN . i-i 

A.  Background . I-l. 

B.  Objective . . . 1-2 

C.  Methodology . 1-3 

II.  SERVICE  PLANS . II-l 

A.  Training  and  Readiness . 11-3 

B.  Costs . ...II-5 

C.  Fiscal  Planning  for  DL . 11-7 

III.  INFORMAL  REVIEW . HI-l 

A.  Cost  Effectiveness . HI-l 

1 .  Computer- Assisted  Instraction  (CAI) . 111-2 

2.  Interactive  Videodisc  (IVD) . ni-2 

3 .  Computer-Based  Instmction  (CBI) . 111-2 

4.  Computer-Based  Training  (CBT) . 111-4 

5  .  Video  Teletraining  (VTT) . 111-4 

B.  Training  and  Readiness . ni-6 

C.  DoDDL  System . .111-7 

D.  Joint-Service  Imperatives . 111-7 

E.  Distance  Learning  in  DoD  Today . 111-9 

F.  Funding  for  DL . III-15 

IV.  DISCUSSION . IV-l 

A.  Interactive  Media . IV-l 

B.  DL  Conversion . IV-l 

C.  VTT  Networks . IV-2 

D.  Conversion  Cost . IV-3 

E.  Cost-Benefit  Analysis . IV-3 


V.  HNDINGS . 

A.  Potential  of  DL . 

B.  DLinDoDToday . 

C.  Up-Front  Investment . 

D.  DL  Conversion  Plans . 

E.  Joint  Service  DL . 

F .  Data  Gaps  and  DL  Investment  Decisions. 

VI.  RECOMMENDATIONS . 

References . 

Glossary . 

Appendix  A — CAI  Evaluation . . . 

Appendix  B — ^IVD  Evaluation . 

Appendix  C — CBI  Evaluation . 

Appendix  D — CBT  Evaluation . 

Appendix  E — VIT'  Evaluation . 


..V-1 

..V-1 

..V-2 

..V-2 

..V-2 

..V-2 

..V-3 

.VI-1 

..R-1 

GL-1 

..A-1 

..B-1 

...C-1 

...D-1 

...E-1 


TABLES 


1 .  Types  of  RC  Training  That  Would  Be  Delivered  by  DL . H- 1 

2 .  PCTcentage  Distribution  of  Planned  RC  DL  Hours  by  Type  Media . II-2 

3 .  FY95  Requirements  for  Training  RC  Enlisted  Personnel  in  Resident  Courses  — 11-3 

4.  Conversion  of  Army  Courses  to  DL . 11-3 

5 .  Number  of  Army  RC  Enlisted  Personnel  Who  Are  Qualified  (Q)  or 

Not  Qualified  (NQ)  in  Their  Duty  MOSs  in  FY95 . . . 11-5 

6.  Number  of  Hours  of  DL  Courseware  for  the  RCs . II-6 

7 .  Cost  to  Convert  All  Resident  Course  Candidates  to  DL  Media  for  the  RCs . 11-6 

8 .  Estimated  FY95  Costs  of  Travel  and  Per  Diem  for  RC  Resident  Training . 11-7 

9 .  Comparison  of  Air  National  Guard  NCO  Academy  Program  Delivered 
in  6  Weeks  in  Residence  or  by  DL  at  Homestations  Followed  by 

2  Weeks  in  Residence . 111-3 

1 0.  Student  Achievement  Improved  and/or  Course  Time  Decreased  When 
Advanced  Training  Technologies  Replaced  Conventional  Instruction 

for  the  Following  Subjects . 111-5 

1 1 .  DoD  Video  Teletraining  Networks . HI- 10 

1 2.  Two-Way  Video,  Two-Way  Audio  (2V,  2A)  VTT  Networks: 

Numbers  of  DoD  Sites . III-ll 

1 3.  One-Way  Video,  Two-Way  Audio  (IV,  2A)  VTT  Networics: 

Numbers  of  Uplinks  and  Downlinks . 111-12 

1 4.  Cumulative  Life  Cycle  Costs  (in  millions  of  U.S.  dollars)  of 

DoD  VTT  Networks  With  93  Sites . HI- 14 

1 5 .  Chimulative  Life  Cycle  Costs  (in  millions  of  U.S .  dollars)  of 

DoD  VTT  Networks  With  15  Sites . 


0 


ni-14 


FIGURES 


1 .  DoD  Distance  Learning  Network  (notional) . 

2.  Locations  of  Uplink  and  DownUnk  Sites  of  Three  DoD  VTT  Networks 


EXECUTIVE  SUMMARY 


The  Deputy  Secretary  of  Defense  tasked  the  Services  on  27  October  1994  to  review 
Distance  Learning  (DL)  technologies!— print,  videotape,  computer-based  training,  inter¬ 
active  videodisc/compact  disc,  and  video  teletraining — ^and  to  submit  plans  by  31  July  1995 
for  using  them  to  train  their  reserve  component  (RC)  personnel.  The  Assistant  Secretary  of 
Defense  for  Reserve  Affairs  subsequently  requested  data  to  assist  ihe  Office  of  the  Under 
Secretary  of  Defense  for  Personnel  and  Readiness  in  evaluating  the  Program  Decision 
Memorandum  (PDM)-directed  Service  plans, 

FINDINGS 

The  following  findings  are  derived  from  a  review  of  the  Service  DL  plans  submitted 
in  response  to  the  Deputy  Secretary  of  Defense  tasking  and  from  the  ongoing  joint-Service 
review  of  training  technology  and  Service  training  requirements  initiated  early  in  1994: 

DL  Has  the  Potential  to  Provide  Effective  Training,  Reduce  Costs,  and 
Improve  Military  Readiness 

•  More  than  300  studies  have  demonstrated  the  efficacy  of  DL  technologies  in 
the  military,  academia,  and  the  commercial  world.  DL  is  as  effective  as 
conventional  resident  instruction  in  achieving  a  wide  range  of  insttuctional 
objectives  (Section  in.A,  Appendices). 

•  Many  cost-effective  applications  of  DL  technologies — ^i.e„  computer-based 
training,  interactive  videodisc,  compact  disc  and  video  teletraining  have 
involved  active  component  (AC)  personnel  and  DoD  civilians  in  schoolhouse 
courses.  Thus,  besides  delivering  DLto  the  RCs,  these  technologies  could  be 
used  to  train  AC  personnel  (1.6  million)  and  DoD  civilians  (900,000)  either  at 
a  distance  of  in  residence  (Section  HI.  A). 

•  DL  can  be  used  to  train  those  who,  for  whatever  reason — usually  a  lack  of 
funds — ^are  unable  to  travel  to  resident  training  sites.  DL  could  be  used  to  train 


!  Distance  Learning  (DL)  is  structured  learning  that  takes  place  without  the  physical  presence  of  an 
instructor.  In  this  paper.  DL  is  a  generic  term  used  to  refer  to  distance  learning,  distance  trsumng, 
distance  education,  distributed  training,  distance  instruction,  etc.  DL  technology  includes  the  full  range 
of  approaches  for  distributing  instruction  to  physically  dispersed  students. 
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the  nearly  46,000  Army  and  Navy  RC  personnel  who  required  training  in 
FY95,  in  contrast  to  the  approximately  20,000  RC  personnel  who  attended 
resident  training  in  that  year  (Section  n.  Table  3). 

•  Fiscal  savings  in  travel  and  per  diem  (which  are  relatively  easy  to  estimate), 
and  reductions  in  time  to  train,  training  material  costs,  instructor  costs,  and 
mafftriflls  preparation  and  modification  costs  potentially  can  be  realized  finm 
use  of  DL  technologies  (Section  n.2). 

•  Increased  readiness  and  workforce  productivity  obtainable  fix)m  compre¬ 
hensive  use  of  DL  technologies  is  difficult  to  estimate,  but  it  may  prove  to  be 
the  primary  payoff  from  systematic  adoption  of  DL  technologies  for  RC 
training  (Section  n.2). 

DL  Is  Little  Used  by  DoD  Today 

•  Investment  reported  by  the  Services  in  DL  technologies  is  limited  and  the  use 
of  these  technologies  is  rare — overall,  estimated  to  be  less  than  1  percent  of 
Service  RC  training  activity  (Section  in.E). 

•  Most  Service  expenditures  for  DL  are  in  hardware  rather  than  courseware,  a 
practice  that  limits  the  utilization  of  DL  systems  (Section  in.F). 

Converting  to  DL  Requires  Sizable  Up-Front  Investments 

•  Conversion  to  DL  courseware  requires  an  up-front  investment  while  resident 
training  programs  continue;  financial  payback  from  reduced  travel,  per  diem, 
and  schoolhouse  costs  begins  as  DL  comes  on  line  and  resident  courses  are 
phased  out  or  reduced  (Section  IV). 

•  Cost  estimates  to  convert  conventional  schoolhouse  instruction  to  DL  course¬ 
ware  vary  widely.  Service  investment  cost  estimates  for  RC  DL  are  sizable, 
ranging  from  about  $5(X)M  to  $1,800M.  However,  eliminating  duplication 
among  the  1,097  courses  that  the  Services  independently  propose  to  convert  to 
DL  would  reduce  these  conversion  costs  (Section  in.2). 

The  Services  Plan  to  Convert  About  One  in  Five  Courses  to  DL 

•  The  Services  plan  to  convert  about  19  percent  of  their  courses  (1,097  of 
5,847)  for  RC  training  to  DL.  Although  the  Navy  plans  to  convert  only 
1  percent  of  its  courses  (27  of  2,433)  for  RC  training,  the  Navy  already  has 
about  600  DL  courses  in  a  print  format  (Section  II). 

•  While  the  Navy  and  Air  Force  plans  indicate  100  percent  conversion  of  all 
prospective  DL  courses,  the  Army  plan  calls  for  100  percent  DL  conversion  of 
only  7  of  532  courses  suitable  for  conversion  (Section  H,  Table  4).  Thus, 


# 
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most  Army  RC  personnel  enrolled  in  DL  courses  would  still  be  required  to 
travel  to  resident  training  sites. 

Joint  Management  of  DL  is  Needed  To  Maximize  Savings 

•  Configuration  standards  are  essential  to  ensure  DL  interoperability  among  the 
Services  and  among  elements  within  eadi  Service  (Section  in.D). 

•  Joint  management  of  DL  courseware  development  is  needed  to  avoid  duplica¬ 
tion  of  course  material;  joint-Service  courseware  can  be  used  in  many  cases  in 
which  Service  courses — whole  or  in  part — ^are  the  same  or  similar  to  those  of 
another  Service  (Section  in.D). 

Data  Gaps  Preclude  Intelligent  DL  Investment  Decisions 

•  Data  furnished  by  the  Services  differ  widely  in  quality  and  comprehensive¬ 
ness.  Despite  commendable  efforts  to  respond  conscientiously  to  the 
DepSecDef  tasking,  gaps  and  uncertainties  remain  in  all  Service  plans  for 
using  DL  in  RC  training  (Sections  U,  IV). 

•  Service  budgets  provide  no  budget  line  items  or  budget  exhibits  under  which 
funding  for  DL  can  be  uniformly  identified,  tracked,  and  managed  (Section  II). 

•  Service  plans  do  not  identify  the  marginal  cost  of  the  schoolhouse  infra¬ 
structure  (instructors,  training  developers,  training  equipment,  course  mainte¬ 
nance,  school  operations,  etc.)  or  discuss  how  implementing  DL  would  affect 
the  present  schoolhouse  infirastmcture  and  costs  (Section  II). 

•  Data  on  opportunities  to  use  DL  technologies  to  train  1.6  million  personnel  in 
the  ACs  and  900,000  DoD  civilians,  eitfier  in  residence  or  at  a  distance,  have 
not  been  compiled  or  studied  (Section  III.C).  Effective  strategic  plans  for  RC 
training  cannot  be  devised  without  full  consideration  of  Total  Force  training 
strategies  and  plans  (Section  IV.E).  With  few  exceptions,  the  content  of 
courses  is  die  same  for  AC  and  RC  personnel  in  the  same  job  occupations. 
Thus,  DL  Courseware  could  be  used  by  the  Total  Force,  whose  larger  popula¬ 
tion  makes  the  economics  of  using  DL  technologies  more  attractive. 

RECOMMENDATIONS 

Based  on  the  above  findings,  senior  representatives  from  the  Office  of  the  Secretaiy 

of  Defense,  the  Joint  Staff,  the  four  Military  Services,  and  other  Defense  Agencies  should; 

1.  Examine  DL  in  a  Total  Force  context:  all  Services,  all  compo¬ 
nents,  and  DoD  civilians.  This  examination  should  include  schoolhouse 
infrastructure  cost,  DL  infrastructure  cost,  travel  and  per  diem  costs,  costs 
associated  with  resident  training,  a  thorough  review  of  the  costs  of  converting 
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conventional  classroom  courses  to  DL  technologies,  and  expected  improve¬ 
ment  in  both  training  efficiencies  and  personnel  readiness. 

2.  Prepare  a  cohesive,  Defense-wide  action  plan  for  DL  that  fully 
capitalizes  on  the  opportunities  provided  by  DL  technologies.  If 
anticipated  improvements  in  training  efficiency  and  personnel 
readiness  warrant,  identify  milestones  that  expeditiously  move 
the  plan  forward  for  implementation.  This  action  plan  should  address 
the  need  for  developing  a  DoD  organizational  stracture  to  assist  in: 

•  Guiding  and  assessing  the  implication  of  DoD  DL  technologies  in  a  Total 
Force  context  that  includes  ACs,  RCs,  and  the  Defense  civilian  work¬ 
force; 

•  Ensuring  that  DoD  DL  technologies  are  developed,  managed,  and  used  in 
a  cost-effective  manner;  and 

•  Identifying  and  enforcing  configuration  standards  for  DoD  DL  tech¬ 
nologies. 
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I.  INTRODUCTION 


A.  BACKGROUND 

The  Services  accomplish  most  of  their  training  of  individuals  (as  compared  to 
training  of  units)^  in  residential  schoolhouse  settings.  This  is  true  for  both  active 
component  (AC)  and  reserve  component  (RC)  training.  For  several  thousands  of  courses, 
the  Services  bear  tiie  travel  costs  and  temporary  duty  (TDY)  costs  needed  to  support  RC 
students  in  centralized  training  facilities.  This  practice  affects  Service  operations,  unit 
performance,  and  quality  of  life  for  individuals  whose  civilian  work  schedule  and  family 
responsibilities  are  interrupted  by  travel  to  distant  training  locations.  These  factors  have 
motivated  the  Services’  RCs  to  look  for  alternatives  to  residential  training  presented  in 
centralized  schoblhouses.  Currently,  the  need  to  identify  acceptable  alternatives  is  increas¬ 
ing  because:  (1)  budget  pressures  are  reducing  funds  to  operate  centralized  schoolhouses; 
(2)  with  modernization  and  technology  enhancements  that  increase  the  complexity  of 
military  equipment  and  support  systems,  training  demands  are  increasing;  and  (3)  national 
strategies  have  increased  dependence  on  reserve  call-up  for  military  operations. 

Representatives  from  the  Services  and  the  Office  of  the  Secretary  of  Defense  (OSD) 
have  been  meeting  quarterly  for  about  2  years  to  discuss  and  promote  the  adoption  of 
Distance  Learning  (DL)^  technologies  by  the  Services.  DL  involves  using  available 
instractional  technologies — i.e.,  print,  videotape,  computer-based  training  (CBT),  inter¬ 
active  videodisc  (IVD)/compact  disc  (CD),  and  video  teletrairting  (VTT) — to  deliver 
training  to  students’  homestations  and  homes.  In  this  sense,  DL  technology  is  not  limited 
to  any  single  instructional  approach,  such  as  VTT.  Some  instruction  is,  of  course,  not 
amenable  to  DL — e.g.,  initial  entry  training  for  new  Service  personnel,  flight  training,  and 


^  “Individual  training”  prepares  the  soldier,  sailor,  airman,  or  marine  in  the  Services’  institutional  base  to 
perftxm  specified  tasks  related  to  his  or  her  assigned  duty  position.  “Unit  training”  (individual, 
collective,  and  joint  or  combined)  takes  place  outside  the  Services’  institutional  base. 

^  Distance  Learning  is  structured  learning  that  takes  place  without  the  physical  presence  of  an  instructor. 
In  this  paper,  DL  is  a  generic  term  used  to  refer  to  distance  learning,  distance  training,  distance 
education,  distributed  training,  distance  instmction,  etc.  DL  technology  includes  the  full  range  of 
approaches  for  distributing  instruction  to  physically  dispersed  students. 


training  that  primarily  concerns  psychomotor  skills — ^but  many  instmctional  objectives  in 
the  Services  can  be  met  by  DL  technologies. 

Two  joint-Service/OSD  review  groups  have  been  formed  to  identify  and  exdiange 
information  concerning  Distance  Learning  technologies: 

•  The  Total  Force  Distance  Learning  Action  Team  is  concerned  with 
lerpiirements  and  technological  opportunities  for  DL^  coherent  and  balanced 
strategies  for  the  adoption  of  DL  and  DL  technologies  by  the  Services; 
ensuring  cost-effective  development,  management,  and  use  of  DL  technol¬ 
ogies;  identifying  standards;  and  ensuring  interoperability. 

•  The  Guard/Reserve  Training  and  Training  Technology  Sub¬ 
committee  establishes  and  oversees  procedures  for  exchanging  infomation 
among  the  Service  RCs  and  relevant  Defense  agencies  about  the  effectiveness, 
costs,  and  availability  of  DL  technology  used  for  education  and  training. 

Formal  tasking  fiom  the  Deputy  Secretary  of  Defense  extended  the  informal  review 
of  DL  requirements  and  technological  opportunities  by  the  Total  Force  DL  Action  Team  and 
the  Guard/Reserve  Subcorrunittee.  Program  Decision  Memorandum  (PDM)  n  of 
27  October  1994  tasked  the  Services  to  review  available  DL  technologies  and  to  subrmt 
plans  for  using  the  technologies  to  train  RC  persotmel  (Ref.  1). 

Additional  guidance  for  the  PDM-directed  study  was  provided  in  a  2  February  1995 
memorandum  to  the  Services  from  the  Assistant  Secretary  of  Defense  for  Reserve  Affairs 
(Ref.  2).  The  memorandum  requested  two  groups  of  data  that  would  facihtate  OSD 
evaluation  of  the  Service  plans.  One  data  set  concerned  measurable  advantages  in  FY95  if 
all  RC  resident  training  for  enlisted  personnel  could  hypothetically  be  delivered  to  the 
students  at  their  home  stations.  The  other  data  set  concerned  course  conversion  criteria, 
multimedia  selection,  course  conversion  cost,  DL  transmit  and  receive  cost,  DL  infia- 
stmcture  cost,  and  schoolhouse  infrastructure  cost. 

B .  OBJECTIVE 

The  objective  of  this  report  is  to  surrunarize  and  evaluate  Service  plans  to  adopt  M- 
technologies.  The  evaluation  includes  consideration  of  information  exchanged  in  the  joint- 
Service/OSD  groups  that  reviewed  training  requirements  and  training  technology. 
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C.  METHODOLOGY 

We  used  documented  Service  plans,  briefings,  training  data,  and  verbal  explana¬ 
tions  provided  by  Service  representatives  to  develop  summary  statistics  for  applying 
technologies  in  RC  training.  Data  from  Service  plans  (Refs.  3,  4,  and  5)  prepared  in 
response  to  the  DepSecDef  tasking  are  discussed  in  Section  n.  Service  Plans.^ 

In  evaluating  the  Service  plans,  we  considered  knowledge  gained  during  a  2-year 
informal  review  of  Service  training  requirements  and  training  technology.  That  knowledge 
is  described  in  Section  m.  Informal  Review.  Six  subsections  discuss  the  following; 

•  The  cost  effectiveness  of  DL  in  numerous  {q>plications. 

•  The  potential  payoff  of  DL  in  improving  individual  readiness. 

•  A  notionaljoint-ServiceDL  system. 

•  Essential  conditions  for  successful  joint-Service  DL. 

•  Current  DL  applications  in  DoD. 

•  Current  and  future  DL  funding. 

Section  n.  Service  Plans,  contains  a  summary  of  Service  plans  to  incorporate  IX 
training.  This  section  also  includes  summary  cost  data.  Section  m.  Informal  Review,  is  a 
commentary  on  die  data  and  summary  statistics  presented  in  Section  n.  Section  IV, 
Discussion,  contains  an  evaluation  of  the  material  presented  in  Section  m. 
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The  Service  plans  documentation  is  too  large  and  unwieldy  to  be  included  as  appendices. 


II.  SERVICE  PLANS 


All  Services  have  policies  that  require  non-prior-Service  personnel  to  attend  recruit 
and  inirial  skill  training  in  residence  at  various  schools.  Tlie  acculturation  requirement 
sarigfiffri  by  this  training  is  the  only  general  principle  stated  by  the  Services  for  excluding 
training  courses  from  DL  presentation.  In  their  plans,  the  Services  indicated  the  types  of 
training  they  propose  for  conversion  to  DL  for  their  RCs.  As  Table  1  shows.  Service  plans 
call  for  DL  {^plications  in  many  types  of  training. 


Table  1.  Types  of  RC  Training  That  Would  Be  Delivered  by  DL 


Type  Training 

Army 

Navy 

Air  Force 

Marine  Corps 

Functional® 

V 

V 

V 

General  Military  Subjects^ 

v 

V 

Professional  Military  Development^ 

V 

V 

V 

V 

Reclassification‘s 

V 

V 

V 

Skill  Progression® 

V 

V 

V 

V 

Special  Skills^ 

yl 

yl 

V 

®  Enables  personnel  to  perform  military  functions,  e.g.,  staff  planning, 
b  For  example,  amphibious  warfare, 
c  Includes  leadership  training. 

^  Initial  skill  training  for  prior-Service  personnel  whose  military  experience  does  not  qualify  them  for  their 
RC  duty  positions. 

®  Prepares  personnel  for  more  advanced  jobs  in  their  fields. 

^  For  example,  hazardous  material  handling. 

Table  2  shows  the  percentage  of  hours  of  DL  media  called  for  by  three  of  the 
Services.  Because  the  Marine  Corps  uses  other  Services’  schools  to  train  over  half 
(52  percent)  of  its  military  occupational  specitdties  (MOSs),  the  DL  media  for  Marine 
training  will  be  some  mix  of  other  Service  choices.  The  Marine  Corps  plans  did  not 
indicate  DL  media  distribution  for  the  training  it  provides  with  its  own  resources. 


Table  2.  Percentage  Distribution  of  Planned  RC  DL  Hours  by  Type  Media 


Service® 

Print 

Videotape 

Computer- 

Based 

Training 

Interactive 

Videodisc/ 

Compact 

Disc 

Video 

Tele- 

training 

Olher^ 

TOTAL 

Army 

19 

12 

27 

21 

11 

10 

100 

Navy 

0^ 

0 

70 

17 

13 

0 

100 

Air  Force 

56 

2 

24« 

18 

0 

100 

®  The  Marine  Corps  will  leverage  other  Services'  DL  since  52  percent  of  Marine  Corps  MOSs  are  trained 
by  the  Army,  Navy,  and  Air  Force. 

*>  Does  not  reflect  about  600  courses  (about  1 ,800  hours)  currently  being  delivered  in  print, 
c  The  Air  Force  treats  CBT  and  IVD  as  a  single  media  category. 

Includes  audio  conferencing. 

Table  3  shows  for  each  Service  the  number  of  courses  (Column  2)  that  could  be 
made  available  to  satisfy  FY95  RC  requirements  for  enlisted  resident  training.  It  also 
shows  the  number  of  courses  (Column  .3)  that  Service  plans  call  for  converting,  in  whole 
or  in  part,  to  DL.  The  Army  would  convert  33  percent  of  its  courses  to  D^-although 
most  would  be  only  partially  converted.  The  Marine  Corps  would  convert  33  percent  of  its 
courses  to  100  percent  DL  deUvery.  The  Air  Force  would  convert  29  percent  of  its  courses 
to  100  percent  DL.  WhUe  the  Navy  would  convert  only  1  percent  of  its  resident  courses,  it 
already  delivers  many  courses  using  DL  technologies.  The  most  used  DL  medium  is  print, 
which  the  Navy  already  uses  for  about  300  correspondence  courses  on  professional/ 
technical  material  and  for  another  approximately  3(X)  correspondence  courses  bn  general 
military  subjects.  These  correspondence  courses  are  prerequisites  for  promotion  examina¬ 
tions  for  all  Navy  enlisted  personnel  (Reserve  and  Active). 

The  fourth  column  in  Table  3  shows  the  number  of  RC  personnel  who  require 
training  in  FY95  and  who  would  receive  it  from  courses  that  the  Service  plans  identify  as 
being  convertible  in  whole  or  in  part  to  DL.  The  fifth  column  shows  the  number  of  RC 
personnel  who  would  receive  training  in  FY95  if  the  courses  the  Services  plan  to  convert 
were  immediately  available  in  DL  format.  No  funds  have  been  budgeted  for  any  of  these 
conversions.  There  is,  therefore,  no  likelihood  that  the  personnel  counted  in  either  the 
fourth  or  fifth  column  will  receive  DL  training.  The  personnel  in  the  fifth  column  were 
scheduled  to  receive  resident  training  in  FY95. 
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Table  3.  FY95  Requirements  for  Training  RC  Enlisted  Personnel 

In  Resident  Courses 


Reserve 

Component 

Number  of  Courses 

Number  of  RC  Personnel: 

Total 

Convertible  Ail  or 

In  Part  to  DL® 

Requiring  Courses 
That  Service  Plans 
Identify  as 
Convertible  to 
Distance  Learning^ 

Who  Would 

Take  in  FY95 

Those  Courses 

That  Would  be 
Converted 

Army  Guard 

1,600 

532  (33%) 

12,606 

8.096  (71%) 

Army  Reserve 

1,600 

532  (33%) 

24,095 

10,104  (42%) 

Navy  Reserve 

2.433 

27  (1.1%) 

9.063 

1.347  (15%) 

Air  National  Guard 

1.400 

402  (29%) 

9,311 

9,311  (100%) 

Air  Force  Reserve 

1,400 

402  (29%) 

3,225 

3.225  (100%) 

Marine  Reserve 

414 

136  (33%) 

.2,120 

2,120  (100%) 

^ '  Funds  to  convert  residerrt  courses  are  not  in  Service  budgets. 

**  Most  Army  courses  would  be  partially  converted;  Army  RC  personnel  in  this  table  require  MOS 
qualification  training  courses,  each  of  which  is  at  least  two-thirds  convertible  to  DL.  All  Navy,  Air  Force, 
and  Marine  courses  would  be  completely  converted  to  DL 

Under  the  plans  summarized  in  Table  3,  the  Navy  would  completely  convert 
27  courses  and  the  Air  Force  would  completely  convert  402  courses  to  DL.  By  contrast, 
the  Army  would  convert  only  7  of  532  courses  to  100  percent  DL,  and  only  2  percent 
(11  courses)  would  be  converted  to  more  than  80  percent  DL.  Table  4  shows  the 
distribution  of  the  Army  DL  courses  by  conversion  percentage. 


Table  4.  Conversion  of  Army  Courses  to  DL 


Percentage  of  Courses 
Converted  to  DL® 

100 

94 

80 

70 

67 

50 

40 

33 

Number  of  Courses 

B 

4 

157 

177 

43 

3 

6 

135 

Percentage  of  532  Courses 

1 

1 

30 

33 

8 

1 

1 

25 

^  Measured  in  hours  of  instruction. 


A.  TRAINING  AND  READINESS 

Table  5  shows  the  training  requirements  for  Army  RC  enlisted  personnel,  who  are 
potentially  the  heaviest  users  of  DL,  both  in  terms  of  the  number  of  MOSs  covered  and  the 
number  of  MOSs  for  which  there  are  resident  training  courses  available  in  FY95.  Data 
from  Table  5  and  the  Army  Training  Requirements  and  Resources  System  (ATRRS) 
suggest  the  training  and  readiness  payoff  that  DL  could  provide  in  FY95: 


•  Resident  training  courses  are  available  for  65  percent  (72,080/1 10,553)  of  all 
Army  RC  enlisted  personnel  who  are  not  MOSQ. 

•  Resident  courses  that  current  Army  plans  call  for  converting  to  at  least 
67  percent  DL  provide  places  for  only  about  one-half  of  the  personnel 
(36,701/72,080)  who  need  them. 

•  ATRRS  data  show  that  FY95  quotas  allow  attendance  by  only  67  percent 
(24,541/36,701)  of  the  RC  personnel  who  need  the  convertible-to-DL  courses. 

•  ATRRS  data  indicate  that  RC  attendees  in  FY94  used  only  about  75  percait  of 
the  quotas  for  resident  course  training  that  were  available  to  them.  (Actual  RC 
attendance  for  FY95'is  not  yet  known.) 

•  18,200  RC  personnel  are  therefore  expected  to  attend  FY95  resident  training 
(0.75  X  24,541). 

•  36,701  personnel  could  be  trained  using  DL  (if  Army  plans  were  budgeted  and 
carried  out)  in  contrast  to  the  18,200  (about  50  percent)  who  instead  are  likely 
to  receive  residential  instmction. 

The  potential  improvement  in  Army  RC  readiness  [twice  as  many  personnel  would 
become  MOSQ  (MOS  qualified)]  fiom  having  the  instmctional  means  to  train  those  who 
would  not  otherwise  attend  resident  training  is  shown  in  the  last  two  columns  of  Table  3 . 
When  both  Army  and  Navy  RCs  are  considered  in  Table  3,  tire  courses  that  their  Services 
plan  to  convert  to  DL  could  potentially  provide  training  to  45,764  RC  personnel  who 
require  individual  training,  whereas  only  19,547  would  attend  the  resident  versions  of 
these  courses  in  FY95.  The  26,217  who  would  not  be  trained  without  DL  conversions 
include  18,501  Army  RC  personnel.  The  potential  gain  is  thus  full  MOSQ  status  for  a 
group  equivalent  in  size  to  about  one-third  of  the  enlisted  strength  of  the  Army  Guard  s 
Enhanced  Readiness  Brigades.  For  the  Navy,  9,063  personnel  require  courses  that  could 
be  converted  to  DL,  but  only  1,347  would  attend  those  courses  in  residence  during  FY95. 
Thus,  converting  these  courses  to  DL  could  provide  training  to  7,716  Naval  Reserve  (NR) 
personnel  who  need  it  but  will  not  receive  it  in  FY95.  On  the  other  hand,  converting  just 
27  Navy  courses  to  DL  raises  the  training  readiness  level  of  the  7,716  NR  personnel  only 
slightly,  from  0.69  to  0.72  in  the  Navy’s  system  for  measuring  Overall  Training  Effective¬ 


ness. 


Table  5.  Number  of  Army  RC  Enlisted  Personnel  Who  Are  Qualified  (Q) 
or  Not  Qualified  (NQ)  in  Their  Duty  MOSs  In  FY95 


Population 

Guard 

Reserve 

Guard  +  Reserve 

Q 

NQ 

Q 

NQ 

Q 

NQ 

NQ 

Q+NQ 

All  Personnel 

251,155 

64,349 

133,461 

46,204 

384,616 

110,553 

.22 

Personnel  with  MOSs  for  which 
there  are  Resident  Training 
Courses  in  FY95 

76,596 

30,206 

120,653 

41,874 

197,249 

72,080 

.27 

Personnel  with  MOSs  who 
could  be  trained  using  resident 
courses  that  are  more  than 

67  percent  converted  to  DL 

28,168 

12,606 

68,779 

24,095 

96,947 

36,701 

.27 

B.  COSTS 

Table  6  shows  the  number  of  hours  of  DL  courseware  called  for  in  the  Service 
plans.  Although  not  budgeted,  the  Service  efforts  would  convert  a  total  of  1,097  RC 
courses — ^532  Army,  27  Navy,  402  Air  Force,  and  136  Marine  Coips — either  wholly  or  in 
part  to  DL.  Service  estimates  of  cost  per  hour  (CPH)  to  convert  resident  course  material  to 
DL  courseware  vary  widely  for  print  ($1,300  to  $5,0(X)),  videou^  ($8,300  to  $17,000), 
VTT  ($260  to  $19,000),  CBT  ($4,000  to  $14,000),  and  IVD/CD  ($4,000  to  $19,000). 
Costs  vary  widely  due  to  such  factors  as  the  nature  of  the  material  being  converted;  the 
quality  of  the  existing  program  of  instruction;  presence  or  absence  of  automated  aids  for 
course  conversion;  tailoring  of  course  content;  sequence,  pace,  and  style  of  instruction  to 
individual  needs;  amount  of  interaction  in  the  conversion  product;  and  different  mixes  of  in- 
house  and  contracted  courseware  development. 

Table  7  shows  that,  based  on  high  and  low  CPH  estimates  by  the  Services,  the  total 
cost  to  convert  to  DL  all  1,097  RC  courses  called  for  by  die  Service  plans  would  range 
from  $526M  to  $1,833M.  To  the  extent  that  duplication  exists  among  the  1,097  courses 
that  the  Services  have  independently  proposed  for  conversion,  an  undetermined  part  of  the 
conversion  cost  can  be  avoided. 


Table  6.  Number  of  Hours  of  DL  Courseware  for  the  RCs 


Distance  Learning  Media 

Number  of  Hours 

Army 

Navy® 

Air 

Force** 

Marine 

Corps® 

All 

Services 

(532 

courses) 

(27 

courses) 

(402 

courses) 

(136 

courses) 

(1.097 

courses) 

Print 

15,932 

0 

8,586 

6,477 

30,995 

Videotape 

10,572 

0 

370 

4,091 

15,033 

Computer-Based  Training 

22,575 

752 

3,643 

9,204 

60,144 

Interactive  Video  Disc/Compact  Disc 

17,564 

184 

7,158 

Video  Teletraining 

9,771 

136 

2,848 

3,750 

32,377 

Other  ^ 

8,653 

0 

0  . 

3,409 

12,062 

Total  Hours  Each  Service 

85,067 

1,072 

15,447 

34,089 

135,675 

a  Aggregated  length  in  hours  of  Navy  RC  courses  assuming  8-hour  days. 


**  The  Air  Force  uses  a  single  media  category  for  CBT  and  IVD. 

®  The  Marine  Corps  wiil  use  the  resources  of  other  Services  for  DL  training.  Total  hours  for  Marine 
training  are  distributed  among  DL  media  using  Army  percentages. 


Table  7.  Cost  to  Convert  All  Resident  Course  Candidates 
to  DL  Media  for  the  RCs 


Cost  per  hour 

Cost  to  Convert 
(thousands  of  dollars) 

(CPH)« 

(dollais) 

High  CPH 

Low 

CPH 

High 

Low 

Army 

Navy 

Air 

Force 

Marine 

Corps 

Army 

Navy 

Air 

Force 

Marine 

Corps 

Print 

5.000 

1,300 

79,660 

0 

42,930 

32,385 

20.712 

11,162 

8,420 

Video  Tape 

17,000 

8,300 

179.724 

0 

6,290 

69,547 

87,748 

0 

3,071 

33,955 

Computer-Based 

Training 

14,000 

316,050 

10,528 

60,110 

128,856 

90,300 

3,008 

14  572 

36,816 

Interactive  Video¬ 
disc/Compact  Disc 

19,000 

4,000 

333,716 

3,496 

136,002 

70,256 

736 

28.632 

Video  Teletraining 

19,000 

260 

185,649 

2,584 

54,112 

71,250 

2,540 

35 

740 

975 

Othofb 

10,000 

10,000 

86,530 

0 

0 

34,090 

86,530 

0 

34,090 

Total  Cost  Each  Servic 

e 

$1,181,000 

$16,608 

$163,442 

$472,130 

$358,086 

$3,779 

$29,545 

$134,468 

Total  Cost  All  Services 

$1,833,180 

$525,878 

*  CPHs  are  highs  and  lows  of  all  Services'  estimates,  except  low  CPH  tor  Video  Teletraining.  $260  is  second  lowest 
CPH  lor  VTT;  Navy  CPH  estimate  is  zero. 


Army  submitted  an  'other  media*  category,  which  is  principally  audio  conferencing,  and  a  single  CPH  estimate. 


# 
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Table  8  shows  current  travel  and  per  diem  expenditures  related  to  TDY  for  central¬ 
ized,  resident  training  for  RC  personnel.  The  average  expenditures  shown  are  (1)  based 
on  historical  Service  data  for  FYs  9(1-94  and  (2)  converted  to  FY95  dollars  by  use  of 
constant  dollar  multipliers.  The  resulting  data  identify  estimated  FY95  RC  travel  and  per 
diem  costs  for  centralized,  residential  training.  The  Service  plans  would  permit  only  a 
portion  of  these  costs  to  be  avoided.  However,  travel  and  per  diem  are  not  die  only 
savings  potendaUy  available  from  use  of  DL  technology  to  provide  RC  training.  Savings  in 
time  to  train,  training  material  costs,  instructor  costs,  instractional  material  preparation 
costs,  and  base  operations  support  will  be  reduced  to  an  unknown  extent  by  implementing 
DL  courses. 


Table  8.  Estimated  FY95  Costs  of  Travel  and  Per  Diem 
for  RC  Resident  Training 


Service 

Average  Cost 
(millions  of  dollars) 

Army 

$32.4 

Navy 

1.2 

Air  Force 

20.1 

Marine  Corps 

1.7 

Total 

$55.3 

The  economic  value  of  increased  training  accessibility  and  the  resulting  increased 
readiness  available  from  more  comprehensive  use  of  DL  technologies  is  difficult  to 
estimate,  but  could  be  greater  than  the  training  cost  savings.  It  may  prove  to  be  the  primary 
payoff  from  systematic  adoption  of  DL  technologies  for  RC  training. 

C .  FISCAL  PLANNING  FOR  DL 

The  Service  DL  plans  reveal  no  budgeting  information  related  to  the  conversion  of 
resident  instruction  to  DL  courseware.  Further,  the  Service  plans  do  not  compare  the 
overall  costs  of  resident  training  versus  DL  using  an  economic  analysis  or  business  plan 
model.  The  great  variability  in  cost  estimates,  reflected  in  Table  7,  suggests  that  a  rigorous 
review  of  costs  is  needed  for  making  decisions  concerning  the  conversion  of  courses  to  IX 
and  the  application  of  DL  in  RC  training. 


III.  INFORMAL  REVIEW 


The  2-year  joint-Service  review  of  training  requirements  and  training  technology 
has  involved  exchanges  of  information  on  ^plicatirais  of  DL  to  many  military  and  civilian 
groups  in  DoD  and  other  federal  agencies  needing  a  variety  of  training  courses.  This  long, 
informal  review  considered  analyses,  pilot  tests,  and  experience — ^in  academia,  the 
commercial  sector,  and  die  military — ^which  demonstrate  that  DL  can  provide  quality 
training  opportunities  at  reduced  cost.  The  substance  of  this  DL  review  is  summarized  in 
the  following  sections:  Cost  Effectiveness,  Training  and  Readiness,  DoD  Distance 
Learning  System,  Joint-Service  Imperatives,  Distance  Learning  in  DoD  Today,  and 
Funding  for  Distance  Learning. 

A.  COST  EFFECTIVENESS 

In  the  late  1980’s,  the  U.S.  Army  Training  and  Doctrine  Command  (TRADOC) 
undertook  an  extensive  study  of  future  Army  training  (Refs.  6  and  7).  For  305  courses 
that  were  recommended  for  full  or  partial  DL,  the  cost  benefit  over  an  assumed  payback 
period  of  7  years  was  $1.52  billion  in  FY90  dollars.  This  payback  represents  potential  net 
savings  resulting  from  subtracting  the  costs  of  course  conversion  and  DL  infrastructure 
from  the  costs  associated  with  formal  resident  instruction  for  the  same  courses  in  TRADCX) 
schools,  academies,  and  centers.  The  TRADOC  study  concluded  that  a  DL  program  would 
“provide  requisite  training  at  a  lower  cost  while  maintaining  training  readiness”  (p.  5-1, 
Ref.  7). 

The  Air  National  Guard  (ANG)  Professional  Military  Education  Center  (PMEC)  is 
partially  converting  to  DL  a  6-week  resident  program  required  for  all  NCOs  in  its  ANG 
NCO  Academy  Program  (Ref.  8).  When  converted  and  validated,  the  same  program  could 
be  delivered  with  only  2  weeks  in  residence  and  DL  for  the  remainder  of  the  course  work. 
Comparative  test  scores  indicate  the  resident-DL  students  are  as  successful  as  die  all¬ 
resident  students.  By  reducing  residency  from  6  weeks  to  2  weeks,  the  resident/DL  course 
will  be  able  to  save  4  weeks  of  per  diem  cost;  will  be  able  to  reach  3,000  students  per  year, 
and  will  be  able  to  solve  a  major  problem  for  Air  Force  RC  NCOs,  who  find  it  difficult  to 
leave  their  civilian  jobs  for  6  weeks. 
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The  NCO  Academy  Program  is  a  prerequisite  for  promotion.  The  ANG  reports  an 
eUgible  population  of  47,000  personnel— too  many  to  be  accommodated  by  resident-only 
instruction— waiting  to  attend  the  NCO  Academy.  Total  Air  Force  requirements  would  add 
thousands  more  potential  students.  This  resident/DL  course  is  currently  under  development 
and  has  not  been  formally  integrated  into  the  ANG  PMEC  program.  FuU  implementation 
would  require  additional  staffing  and  instructor  resources.  Table  9  shows  a  cost  compari¬ 
son  and  potential  savings  (in  constant  dollars)  ftom  this  partial  conversion  of  the  NCO 
Academy  Program  to  DL. 

The  appendices  sumrnarize  reviews  of  over  300  studies  that  evaluated  the  effective¬ 
ness,  in  terms  of  student  achievement,  training  cost,  and  reduction  in  student  training  time, 
of  various  DL  technologies  compared  to  conventional  mstruction. 

1.  Computer-Assisted  Instruction  (CAl) 

Orlansky  and  String  (Ref.  9,  and  see  Appendix  A)  found  that  (1)  student  achieve¬ 
ment  with  CAI  and  computer-managed  instruction  (CMI)  courses  in  military  training 
schools  was  the  same  or  greater  than  with  conventional  instruction,  and  (2)  CAI  and  CMI 
saved  about  30  percent  of  the  time  (median  value)  needed  by  smdents  to  complete  the  same 
courses  given  by  conventional  instruction.  The  findings  are  tire  result  of  48  comparisons 
of  CAIAIIMI  and  conventional  instruction  in  30  different  studies. 

2.  Interactive  Videodisc  (IVD) 

In  47  comparisons  of  IVD  and  conventional  instruction  in  various  settings— 
namely,  military  training,  industrial  training,  and  higher  education — ^Fletcher  (Ref.  10,  and 
see  Appendix  B)  found  that  IVD  increased  achievement  for  the  average  smdent  (50th 
percentile)  to  the  69th  percentile.  The  average  amount  of  time  saved  by  students  using  IVD 
in  all  these  comparisons  was  31  percent. 

3.  Computer-Based  Instruction  (CBI) 

Redding  and  Retcher  (Ref.  11,  and  see  Appendix  C)  presented  results  from 
numerous  studies  of  CBI  showing  the  foUowing  increases  in  percentile  achievement  for  the 
average  student  in  three  instractional  settings:  65th  percentile  in  42  studies  of  secondary 
school  students;  60th  percentUe  in  101  studies  of  college  students;  66th  percentile  in  24 
studies  of  adult  education  students. 
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Table  9.  Comparison  of  Air  National  Guard  NCO  Academy  Program 
Delivered  in  6  Weeks  in  Residence  or  by  DL  at  Homestatlons 
Followed  by  2  Weeks  In  Residence  (Source:  Ref.  8) 


A.  BACKGROUND  DATA 


•  DL  involves  thirty-seven  3-hour  VTT  sessions;  DL  +  2  weeks  In  residence  Includes  same 
course  content  as  6  weeks  In  residence. 

•  VTT  network  includes  93  DL  site  facilities  (homestatlons)  with  12  students  per  site  (total  of 
1,116  students). 

•  VTT  equipment:  1  -way  video.  2-way  audio  acquired  by  purchase  based  on  one  channel  uplink. 

•  Each  student  gets  $29  for  each  3-hour  DL  session  and  $1 ,060  for  2  weeks  in  residence;  each 
student  gets  $3,532  for  6  weeks  in  residence. 

•  Travel  cost  to  resident  training  facility:  $392. 

•  Facilitators  monitor  DL  site  training  and  provide  pinch-hit  instruction  in  case  of  VTT  failure; 

2  facilitators  are  trained  for  each  DL  site. 

•  Facilitator  training  cost  is  $1 ,205  each;  facilitator  pay  per  3-hour  DL  session  is  $35. 


B.  FIRST  YEAR  COSTS  (thousands  of  dollars) 


Expense  Item 


EQUIPMENT 

Compressed  Digital  Video  (nonrecurring)-l  V.  2A* 
Communications  (recurring) 

Operation  and  Maintenance  frecurrina) 


STUDENTS 
Pay  for  DL  phase 
Pay  for  Resident  phase 
Travel 


FACILITATORS 
Pay  while  in  training 
Pay  for  DL  sessions 
Travel 


Totals 


6-Week  Residence  |  DL  Plus  2-Week  Residence 


$4,378 


224 

120 

73 


$5,202 


Redding  and  Fletcher  also  describe  in  Appendix  C  a  report  of  about  250  primary 
studies  that  provide  quantified  measures  of  student  achievement  in  several  instmctional 
settings— namely,  elementary  school,  secondary  school,  coUege,  Japanese  schools,  and 
special  education.  Overall  in  these  studies,  CBI  raised  the  percentile  achievement  of  the 
average  smdent  to  the  66th  percentile.  The  report  also  covered  about  100  CAI-versus- 
conventional  instmction  smdies  that  did  not  use  quantified  measures  of  student  achieve¬ 
ment;  these  studies  do  conclude,  however,  that  CAI  enables  students  to  learn  as  much  in 

less  time. 

4.  Computer-Based  Training  (CBT) 

Johnston  and  Fletcher  (Appendix  D)  reviewed  33  empirical  studies  that  compared 
CBT  with  conventional  instmction  in  military  training  (Ref.  12).  Thirteen  of  the  33 
comparisons  are  included  in  the  48  comparisons  considered  in  Appendix  A.  However, 
although  student  achievement  was  not  quantified  in  the  Appendix  A  comparisons  of 
CAVCMI  and  conventional  instmction,  all  33  comparisons  in  the  Johnston-Fletcher  review 
use  percentile  achievement  to  measure  the  effectiveness  of  instmction.  Johnston  and 
Fletcher  found  in  the  33  comparisons  that  CBT  raised  the  average  student’s  achievement  to 
the  67th  percentile.  In  a  subset  of  23  comparisons  (11  of  which  appear  in  Appendix  A)  that 
measured  instmction  time,  average  time  savings  from  using  CBT  was  30  percent;  that 
result  is  consistent  with  average  time  saving  reported  in  Appendix  A. 

5.  Video  Teletraining  (VTT) 

The  commercial  world  and  academia  increasingly  use  VTT  to  reduce  travel,  per 
diem,  and  instmctional  delivery  costs.  Several  studies  that  are  summarized  in  Appendix  E 
indicate  that  VTT  can  be  at  least  as  effective  as  residential  training  in  military  applications 
and  that  it  could  be  less  costly  as  well. 

Table  10  lists  subjects  of  military  courses — ^varying  in  length  from  a  few  days  to 
several  weeks— hi  which  student  achievement  improved  or  course  time  decreased  or 
both— when  advanced  training  technologies  replaced  conventional  instmction. 


table  10.  Student  Achievement  Improved  and/or  Course  Time 
Decreased  When  Advanced  Training  Technologies  Replaced 
Conventional  Instruction  for  the  Following  Subfects 


Computer-Assii 

sted  Instruction 

Basic  Electronics 

Medical  Assistant 

Electricity 

Vehicle  Repair 

Machinist 

Weather 

Training  Materials  Development 

Tactical  Coordinator  (S-3A) 

Recipe  Conversion 

Airaaft  Panel  Operation 

Fire  Control  Technician 

Computer-Managed  Instruction 

Aviation  Familiarization 

Materiel  Facilities 

Aviation  Mechanical  Fundamentals 

Precision  Measuring  Equipment 

Inventory  Management 

Weapons  Mechanic 

Computer-Based  Trainini 


Basic  Reading  Skills 

Copilot  Integrated  Control  System 

Medical 

Vehicle  Repair  (cooling  system,  warning 
system,  clutch,  brakes) 

Troubleshooting  Logic  Circuits 

Electronic  Principles 

Air  Traffic  Control — ^Weather  Fundamentals 

Fuel  Savings  Advisory  System 

Basic  Electronics 

Recipe  Conversion 

Keyboard 

Developmental  Tactical  Operations  System 
Troubleshooting  M1A1  Fuel  Supply  System 

Interpersonal  Skills 

Machinist  Course 

Personnel  Administration 

Crew-Served  Weapons 

Console  Operation 

Foreign  Language 

Operation  and  Maintenance  6883  Test  Station 
Typing 

Oscilloscope  Operation 

Electronic  Technician 

Tactical  Coordinator 

Acoustic  Analysis 

Reading  Skills 

Interactive  Videodisc 

Interpersonal  Skills 

Foreign  Language  (Beginning  Spanish) 

Helicopter  Maintenance  (hydraulics) 

Physics  “Puzzle  of  the  Tacoma  Bridge 

CPR  Recertification  Training  for  Registered 

Collapse” 

Nurses 

Biology  (Respiration) 

Communication  Circuit  Maintenance 

Biology  (Climate  and  Life) 

Handling  of  Hazardous  Materials 

Chemistry  (Kinetics  and  Equilibrium) 

Equipment  Operation  (16-mm  projector) 

Chemistry  (Gas  Analysis) 

Biology  “Development  of  Living  Things’ 

Medical  Education  (Trauma) 

Foreign  Language  (Beginning  French) 

Public  Health  (CPR  Instruction) 

Video  Teletrainini 


Maintenance  Record-Keeping 

Army  Command  and  General  Staff  College 

Safety 

Courses 

Divisional  Chief  Petty  Officer  Indoctrination 
Common  Leadership  Training 

Logistics  Management 
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B.  TRAINING  AND  READINESS 

Among  other  activities,  the  Total  Force  DL  Action  Team  and  the  Guard/Reserve 
Subcommittee  reviewed  individual  training  and  readiness  in  the  Army  Guard  and  the  Army 
Reserve.  The  following  statistics  pertain  to  190,000  Army  RC  enlisted  personnel  in  81 
military  occupational  specialties  for  combat  support  and  combat  service  support  (Ref.  13). 

•  63  percent  of  Total  Army  personnel  belong  to  the  RCs. 

•  39  percent  of  the  RC  personnel  were  not  MOSQ  for  their  duty  positions; 
4  percent  of  counterpart  personnel  in  the  Army  AC  were  not  MOSQ.^ 

.  Only  41  percent  of  the  Army  RC  personnel  who  were  not  MOSQ  were  slated 
to  attend  resident  training  in  FY95.  Lack  of  funding  is  the  principal  reason  for 
not  sending  59  percent  of  non-MOSQ  RC  students  to  resident  training. 

Army  RC  units  use  mentors,  who  are  diverted  from  their  normal  duties,  to  provide 
whatever  training  can  be  given  locally  to  the  59  percent  of  RC  personnel  who  are  not 
MOSQ  and  not  scheduled  for  resident  training.  The  availability  of  DL  would  (1)  enable  a 
significant  portion  of  the  RC  personnel  who  cannot  go  to  resident  courses  to  obtain  some 
MOSQ  training  and  (2)  save  the  travel  and  per  diem  expenses  of  sending  41  percent  of  RC 
persoimel  away  for  training. 

For  a  readiness  perspective  on  MOSQ  percentages,  consider  different  mihtary 
operations  involving  call-up  of  Reserves.  For  small  or  moderately  sized  operations,  such 
as  Haiti,  there  are  a  couple  of  options  for  mobilizing  a  well-trained  and  ready  Reserve 
contingent  even  if  large  parts  of  the  RCs  are  not  MOSQ.  First,  only  \mts  stocked  with 
MOSQ  personnel  would  be  selected  for  call-up.  Second,  derivative  subunits  created  from 
larger  units  can  be  combined  with  individual  transfers  to  form  new  units  heavy  in  MOSQ 
personnel. 

For  a  large  operation,  such  as  the  Gulf  War,  where  the  Reserve  caU-up  is  also 
large,  having  a  high  percentage  of  RC  personnel  who  are  not  MOSQ  is  a  liability  and  could 
jeopardize  the  operational  mission.  Loading  up  a  few  units  with  MOSQ  personnel  and 
shuffling  subunits  and  individuals  to  form  new  units  are  not  useful  options  when  training 
deficiencies  are  widespread  and  mobilization  time  is  short. 


5  To  become  MOSQ,  a  soldier  must  demonstrate  in  a  formal  course  of  instruction  proficiency  in 
individual  skills  and  an  ability  to  do  his  or  her  job. 
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C.  DoD  DL  SYSTEM 


A  concept  for  a  DL  system  was  formulated  in  die  meetings  of  the  Total  Force  IX. 
Acticni  Team  and  the  Guard/Reserve  Subcommittee.  This  concept  for  a  joint-Service 
tqiproach  to  the  evolution  of  DL,  including  the  ACs  with  their  1.6  milhon  personnel,  is 
shown  in  Figure  1.  This  Total  Force  DL  plan  could  include  DL  for  900,000  DoD  civilians, 
about  half  of  whom  receive  about  50  hours  of  training  aimually  at  a  cost  of  about  $125M 
for  travel  and  per  diem.  A  Total  Force  DL  Plan  could  also  provide  the  means  for  an 
interface  with  VTT  networks  of  other  Federal  agencies  for  mutual  advantage  and  savings. 

D.  JOINT-SERVICE  IMPERATIVES 

Early  in  the  joint-Service  review,  it  was  recognized  tiiat  (1)  configuration  standards 
were  required  to  ensure  DL  interoperability  among  the  Services,  and  (2)  economies  in 
courseware  development  must  be  sought  for  DL  courseware  useful  to  more  than  one 
Service.  The  configuration  standards  issue  has  three  dimensions:  media  selection,  tech¬ 
nical  performance,  and  economics. 

•  Mpidia  selftctinn:  Services  must  adopt  the  same  media,  i.e.,  print,  videot^ie, 
computer-based  training,  interactive  videodisc/compact  disc,  and  video 
teletraining,  for  a  common  course  or  common  segment  of  a  course.  This 
requirement  would  not  preclude  joint-Service  adoption  of  more  than  one 
medium  to  fit  different  situations  among  the  target  populations  to  be  trained. 
For  example,  VTT  might  be  used  to  deliver  training  to  DL  centers,  while  the 
same  instructional  material  contained  on  CD  can  be  used  at  overseas  bases,  on 
ships  at  sea,  or  with  personal  computers  in  the  home. 

•  Technical  performance:  Baseline  technical  performance  criteria — e.g.,  frame 
rate  for  video  imagery  and  transmission  rate  for  VTT  networks,  and  ME^ 
STD-1379D  for  interactive  courseware  (ICW)  (Ref.  14) — ^must  be  used  by  all 
Services. 

Economics:  A  Service’s  choice  of  DL  system  architecture  and  communication 
network  should  not  penalize  die  other  Services  with  needless  costs.  Fielding 
DL  courseware  products  or  services  involving  closed  system  architectures 
defeats  the  ability  to  leverage  resources  by  effectively  denying  access  to  those 
outside  the  system. 
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Figure  1.  DoD  Distance  Learning  Network  (notionai) 
(Source:  Army  Training  Support  Center  Briefing) 


®  Uplink  Site/Transmit  Station  •  Downlink  site/Leaming  Center 

Fxistinn  Facilities:  Military  bases.  Reserve  centers,  and  Guard  armories  will  be 
the  DL  Centers  (site  dots  on  map  do  not  represent  actual  facilities). 

Mfidia  Available:  VT,  VTT,  CBT,  IVD/CD  equipment  developed  to  common 
standards. 

Users:  All  components  of  all  Services;  some  sharing  of  DL  Centers  between 
active  and  reserve  components  and  among  Services  will  occur. 

Training  Focus:  Cognitive  Skill  training  at  the  DL  Centers;  psychomotor  skill 
training  will  be  done  during  monthly  inactive  duty  training  and  annual  active 
duty  training  periods. 

Numerous  PI  Centers:  Enough  that  most  Service  personnel  live  within 
50  miles  of  one  so  that  travel  and  per  diem  costs  would  be  avoided.  Students 
with  computer  and  visual  information  equipment  could  get  stay-at-home 
training. 

Student  Suonort  Groups:  Formal  or  informal  peer  groups  could  be  formed 
for  DL  courses  as  they  are  for  conventional  schoolhouse  courses  to  facilitate 
learning.  _  _ 


Training  courses  for  many  occupations  common  among  the  Services  (e.g., 
electrician,  tmck  mechanic,  aircraft  maintenance,  medical  personnel,  food  service,  ord¬ 
nance  handling,  motor  transport  operator)  have  essentially  the  same  instractional  content, 
regardless  of  the  Service.  Duplication  of  joint-Service  courseware,  the  development  of 
which  is  very  expensive,  must  be  avoided. 


E.  DISTANCE  LEARNING  IN  DoD  TODAY 


Interactive  media  applications — i.e.,  CBT,  IVD,  and  CD — ^are  rarely  used  in 
Defense  training.  They  represent  a  mature  technological  opportunity  that  is  seldom  used. 
These  media  have  been  shown  by  empirical  evidence  from  numerous  applications  of 
interactive  courseware  to  reduce  training  time  by  about  30  percent  (see  Sections  III.A.1, 4), 
and  to  improve  average  student  percentile  achievement  from  50  to  60  or  more  (see  Sections 
ra.A.2,  3,  4). 

The  Services  use  a  mix  of  in-house  resources  and  contractor  support  to  develop  IX 
courseware.  The  selection  of  DL  media  is  based  on  familiar  training  models,  instructional 
designs,  and  network  architectures.  Although  there  is  no  formal  tracking  of  media 
utilization,  no  Service  representative  estimated  drat  more  than  1  percent  of  DoD  individual 
training  is  currently  being  delivered  by  DL  technologies. 

The  introduction  of  VTT  within  the  DoD  grew  largely  from  pilot  programs  directed 
by  the  Army.  Funds  for  VTT  from  various  defense  appropriations  have  been  directed 
primarily  at  Army  Reserve  and  Army  National  Guard  (ARNG)  training.  Data  collected 
from  these  pilot  programs  and  technology  demonstrations  were  later  used  by  the  Navy  and 
Air  Force  to  develop  their  own  implementations  of  VTT.  The  Navy  designed  its  VTT 
network  to  extend  school  and  resident  training  capabilities,  but  CESN  [CNET  (Chief  of 
Naval  Education  and  Training)  Electronic  Schoolhouse  Network]  sites  have  not  been 
established  solely  for  NR  training.  The  Air  Force  designed  and  implemented  three  VTT 
networks  to  serve  the  Active,  Reserve,  and  Guard  components  of  the  Air  Force,  each  using 
its  own  funds  and  needs  assessment.  Thus,  these  three  Services  have  three  different 
focuses  for  VTT  applications:  Air  Force  AC  and  RC,  Navy  AC,  and  Army  RC.  The 
Army’s  RC  focus  for  VTT  applications  continues  through  congressionaUy  directed  studies 
and  pilot  programs  that  target  the  ARNG  as  a  VTT  testbed.  Based  on  full  VTT 
implementation  by  the  other  Services,  there  is  no  technological  or  operational  requirement 
for  the  Army  to  continue  to  field  VTT  systems  as  pilot  programs  or  testbeds. 

Within  the  DoD,  the  Services  operate  six  separately  managed  VTT  networks.  Each 
network  has  the  means  to  send  and  receive  DL  courseware  using  internal  network 
resources.  Defense  Agencies,  on  the  other  hand,  are  establishing  networks  to  “piggy¬ 
back”  Services’  networks  or  obtain  DL  programming  from  soiuces  outside  DoD.  DoD 
networks  do  not  employ  a  single  distribution  architecture,  but  with  the  proper  use  of  gate¬ 
ways  there  is  no  compelling  reason  to  do  so,  either  to  share  DL  materials  or  to  achieve 
economies  of  scale.  Table  11  identifies  the  six  VTT  networks  used  by  the  Services. 


Table  11.  DoD  Video  Teletraining  Networks 


Network 

Video,  Audio  Capability 

Chief  of  Naval  Education  and  Training  (CNET) 
Electronic  Schoolhouse  Network  (CESN) 

2V,  2Aa 

Satellite  Education  Network  (SEN) — Army 

IV,  2A‘’ 

Training  Network  (TNET) — ^Army 

2V,  2A 

Air  Technology  Network  (ATN) — ^Air  Force 

1V.  2A 

Air  Force  Reserve  Network  (TNET-Based) 

2V,  2A 

Warrior  Network — Air  National  Guard 

1V,  2A 

^  2-way  video,  2-way  audio. 
^  1  -way  video,  2-way  audio. 


VTT  networks  may  model  commercial  broadcasting  (1-way  video)  or  video 
teleconferencing  (VTC)  (2-way  video).  There  are  significant  differences  in  what  the 
technologies  can  do,  given  the  realities  of  a  distributed  classroom  environment.  In  order  to 
achieve  meaningful  interactivity  with  remote  sites  the  number  of  locations  that  can  be 
accommodated  in  a  single  session  must  be  linuted.  For  example,  while  it  may  be  tech¬ 
nically  possible  to  reach  over  100  remote  locations  using  1-way  video  (IV)  with  30 
students  at  each  site,  it  is  not  reasonable  to  expect  any  meaningful  instructional  dialogue 
finm  such  an  extensive  network.  Far  fewer  sites  arc  typically  involved  in  a  2-way  video 
(2V)  VTT  session.  Based  on  industry  data,  about  50  percent  of  all  VTC  events  involve 
only  two  sites,  and  events  with  four  sites  or  less  account  for  85  percent  of  all  VTC. 
Unfortunately,  network  sponsors  are  prone  to  emphasize  the  technological  “reach”  of  the 
system,  when  the  focus  might  be  better  placed  on  quality  of  the  DL  instraction  provided, 
accessibility  by  students,  and  student  throughput. 

Table  12  shows  the  number  of  DoD  VTT  2V,  2A  sites.  Although  the  classroom 
hardware  systems  for  these  networks  is  similar,  the  Army’s  TNET  is  a  satellite-based 
network  whereas  the  Navy’s  CESN  uses  terrestrial  FrS20(X)  and  Integrated  Services 
Digital  Network  (ISDN)  circuits,  both  enable  of  transmitting  at  384  Kbps.  Differences  in 
audio  systems  preclude  direct  interoperability  from  a  TNET  site  to  a  CESN  site.  However, 
the  CESN  network  hub  at  Dam  Neck,  Virginia,  is  able  to  operate  as  a  gateway  between  the 
two  networks. 


Table  12.  Two-way  Video,  Two-way  Audio  (2V,  2A)  VTT 
Networks:  Numbers  of  DoD  Sites" 

(Source:  Network  Managers) 


Network 

Sites 

FY95 

Projected 

Navy  CESN 

13 

25‘> 

Air  Force  Fteserve  TNET 

47 

47 

Army  Training  Support  Center  TNET 

68 

68 

TOTAL 

129 

144 

"  Each  site  has  transmit  and  receive  capability. 

^  Does  not  include  shipboard. 

Table  13  shows  current  and  projected  numbers  of  VTT  downlink  sites  using  IV, 
2A  networks  that  belong  to  DoD  and  other  federal  agencies.  The  DoD  networks  have  been 
acquired  through  contracts  issued  through  the  Defense  Information  Systems  Agency 
(DBA).  Federal  agencies,  including  DoD,  can  obtain  similar  equipment  and  services  from 
GSA’s  FTS2000  contract.  Both  contracts  offer  parallel  systems  and  services. 

Downlinks  are  shown  in  Table  13  without  differentiating  between  analog  and  digital 
signal  recepticMi  capabilities.  Analog  C-band  and  Ku-band  broadcasts  require  a  larger 
bandwidth  to  transmit  signals  than  for  compressed  digital  video  (CDV)  broadcasts.  Ten 
CDV  chaimels  can  be  simultaneously  broadcast  within  the  same  Ku-bandwidth  required  for 
a  single  analog  chaimel.  The  resulting  cost  economies  and  scarce  C-band  satellite 
transponder  availability  are  prompting  network  managers  to  look  at  uplinking  VTT 
digitally.  Receive-only  dishes  can  be  configured  to  accept  either  transmssion  mode. 
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Table  13.  One-way  Video,  Two-way  Audio  (IV,  2A)  VTT  Networks: 

Numbers  of  Uplinks  and  Downlinks*  _ 


Downlinks 

Networks.  Uplinks 

FY95  1 

1  DoD 

Air  National  Guard  Warrior  Network,  3  uplinks 

93 

240 

Air  Force  Reserve 

0 

5 

Air  Technoloov  Network  (ATN),  3  uplinks 

72 

72 

Armv  National  Guard,  no  uplink'* 

0 

30 

Defense  Information  Systems  Agency,  no  uplink® 

22 

22 

Armv  Satellite  Education  Network  (SEN),  multiple  uplinks*' 

83 

83 

TOTAL 

280 

452 

1  Federal  Agencies 

10 

20 

Environmental  Protection  Agency 

70  .  ^ 

70 

Federal  Aviation  Administration,  1  uplink 

38 

392 

Veterans  Benefits  Administration 

0 

60 

78 

272 

United  States  Postal  Service,  4  uplinks,  4  channels® 

300 

350 

United  States  Coast  Guard 

0 

20 

Social  Security  Administration 

300 

331 

*  Source:  AT&T  TRIDOM.  All  1  -way  video,  2-way  audio,  and  subject  to  change. 

Based  on  a  proposed  plan  to  access  SEN  and  ANG  Warrior  network, 
c  Installed  with  optional  NTU  equipment  to  access  fee-for-service  courseware. 

SEN  is  the  only  DoD  “gateway”  capable  of  receiving  and  retransmitting  C-band,  Ku-band,  and  CDV. 

«  C-band  and  Ku-band. 


Figure  2  shows  a  U.S.  map  with  actual  locations  of  uplink  and  downlink  sites  of 
three  IV,  2A  DoD  VTT  networics;  Air  Force’s  Air  Technology  Network  (ATN);  the 
Army’s  Satellite  Education  Network  (SEN);  and  Air  National  Guard’s  Warrior  Network 
(WN).  Other  fee-for-service  educational  networks  that  can  be  accessed  by  DoD  s  1 V,  2A 
locations  using  compressed  digital  video  (CDV)  systems  are  identified  below: 

.  National  Technological  University  (NTU):  dehvers  25,000  programs  annually 
from  46  universities;  deUvers  13  MS  degree  curricula  and  over  1,000  graduate 

courses. 

.  PubUc  Broadcasting  System  Adult  Learning  Service:  provides  access  to  2,000 
colleges  and  universities. 

•  Satellite  Education  Resource  Center:  includes  500  schools  in  28  states. 


•  Satellite  Education  Offices:  includes  offices  in  Honda,  Georgia,  Louisiana, 
and  South  Carolina. 

Figure  2.  Locations  of  Uplink  and  Downlink  Sites  of 
Three  DoD  VTT  Networks" 


DoD  DL  Sites  (JV,2A) 


^  Source:  Air  Force  institute  of  Technology  Center  for  Distance  Education. 
The  DoD  does  not  maintain  a  consolidated  database  of  VTT  sites. 


The  economics  of  VTT  networks  for  existing  and  planned  applications  illustrates 
the  need  for  DL  configuration  standards.  As  shown  in  Table  9,  the  acquisition  of  a  CDV 
IV,  2A  VTT  network  architecture  by  the  Air  National  Guard  (ANG)  provides  affordable 
scalability  for  the  DL  phase  of  its  NCO  Academy  Program.  The  ANG’s  WN  with  its  basic 
CDV  architecture  can  be  scaled  to  any  number  of  receive  sites.  On  the  other  hand,  the  2V, 
2A  architecture  of  CESN  and  TNET  would  present  technological  problems  in  concurrently 
teaching  93  ANG  sites  for  common  training.  In  their  basic  configurations,  CESN  and 
TNET  are  constrained  by  technical  limitations  to  22  and  15  sites,  respectively,  vitithout 
additional  hubs.  However,  additional  multipoint  control  units  can  be  “cascaded”  to  extend 
the  number  of  participating  sites.  Table  14  shows  the  cumulative  5-year  life  cycle  cost  if 
CESN  and  TNET  network  architectures  were  leased  and  used  to  reach  93  sites  for  the  NCO 
Academy  Program. 


Table  14.  Cumulative  Life  Cycle  Costs  (In  millions  of  U.S.  dollars) 
of  DoD  VTT  Networks  with  93  Sites  (Source:  Ref.  15) 


Lease  (L)  or 
Purchase  (P) 

Year 

Network 

1 

2 

3 

4 

5 

CESN-2V,  2A 

L 

6.234 

12.467 

18.701 

24.934 

31.168 

TNET-2V.  2A 

L 

9.368 

17.421 

25.474 

33.527 

41 .580 

CDV-1V,2A 

P 

3.002 

4.083 

5.164 

6.245 

7.325 

The  costs  in  Table  14  are  driven  by  a  requirement  to  deliver  instructional  material  to 
many  students  (1,116)  at  many  sites  (93).  As  Table  15  shows,  if  the  NCO  Academy 
Program  were  delivered  to  only  15  sites  (with  12  students  each),  the  advantage  of 
purchasing  IV,  2A  VTT  equipment  over  leasing  2V,  2A  equipment  is  much  less  marked. 
(The  table  also  shows  that  CESN  has  lower  life-cycle  costs  than  TNET,  as  it  did  in  the 
93-site  case).  While  a  2V,  2A  system  is  needed  for  VTC,  IV,  2A  networks  are  generally 
the  cost-effective  choices  for  training,  because  the  two-way  audio  satisfies  nearly  all 
training  needs  for  student-instractor  interaction.  Moreover,  IV,  2A  VTT  systems  are  less 
costly  to  expand  to  deliver  training  to  additional  sites  (scalability).  However,  when  the 
Navy  uses  VTT  for  the  NR,  it  typically  delivers  courses  to  a  few  students  at  a  few  sites  and 
finds  its  leased  2V,  2A  system  to  be  less  costly  than  purchasing  IV,  2A  equipment.  Thus, 
both  current  training  requirements  and  scalability  are  important  considerations  in  selecting  a 
VTT  network.  Decisions  to  lease  or  purchase  VTT  hardware  (all  VTT  network  circuits  are 
leased)  are  based  on  several  considerations:  the  cost  of  equipment  and  software  upgrades, 
the  cost  of  maintaining  the  facilities,  the  type  of  funding  available,  and  the  ability  to 
adequately  staff  and  manage  a  network  operation. 


Table  15.  Cumulative  Life  Cycle  Costs  (In  millions  of  U.S.  dollars) 
of  DoD  VTT  Networks  with  15  Sites 


Network 

Lease  (L)  or 
Purchase  (P) 

Year 

1 

2 

3 

4 

5 

CESN-2V,  2A 

L 

1.043 

2.085 

3.128 

4.171 

5.213 

TNET-2V.  2A 

L 

1.495 

2.840 

4.335 

5.830 

7.325 

CDV-IV,  2A 

P 

1.049 

1.418 

1.787 

2.155 

2.524 

Joint-Service  use  of  a  VTT  network  could  involve  changes  in  equipment  and/or 
software  to  accommodate  training  requirements  beyond  those  of  the  network  owner- 


operator.  If  modifying  an  established  network  is  the  least  costly  alternative  to  satisfy  other 
Services’  requirements,  it  seems  reasonable  that  die  cost  of  network  modification  be 
charged  to  the  new  user-Services.  For  example,  tire  Navy  plan  has  the  2V,  2A  CESN 
network  using  smaller  bandwidth  circuits  and  desktop  (as  opposed  to  classroom)  systems 
for  197  NR  sites.  As  a  consequence,  the  CESN  transmission  rate  of  384  Kbps  would  be 
reduced  to  128  Kbps  for  NR  training.  While  some  audio  and  motion  video  quality  is 
sacrificed,  the  Navy  believes  die  quality  is  suitable  for  its  NR  training  requirements.  A 
draft  DoD  instraction  on  VTT  networks  has  identified  384  Kbps,  full  common  intermediate 
format  (CIF),  as  the  minimum  acceptable  transmission  speed  for  DoD  DL  networks.  If 
Services  elect  to  use  less  than  384  Kbps  for  internal  training  requirements,  they  would 
logically  be  expected  to  incur  the  cost  of  upgrading  to  384  Kbps  when  participating  in  joint- 
Service-user  DL.  The  point  is  that  joint-Service-user  business  decisions  on  system 
architecture  and  communication  networks  are  necessary  to  achieve  DL  interoperability 
among  the  Services. 

F.  FUNDING  FOR  DL 

Assessing  the  investment  in  DL  against  a  potential  payback  defies  a  single  solution. 
Each  Service  has  developed  its  own  criteria  for  funding  DL,  but  few  initiatives  have  been 
preceded  by  formal  needs  assessment  or  requirements  analyses.  Early  work  by  the  Army 
in  pilot  VTT  studies  dealt  mostly  with  the  feasibility  of  the  technology  and  did  not  examine 
life  cycle  management.  None  of  the  Defense  Agencies  or  Services  are  obligated  to  submit 
Program  Objective  Memorandum  (POM)  actions  to  support  DL  programs.  One  reason  is 
that  funding  for  DL  programs  has  been  “taken  out-of-hide”  by  the  training  communities  as 
they  strive  to  avoid  the  costs  of  resident  instmction.  While  this  may  be  acceptable  for  low- 
tech  DL  tedmologies  such  as  publishing  courseware  on  compact  disc-read-only  memory 
(CD-ROM)  or  establishing  “chat”  lines  on  the  Internet,  more  stringent  criteria  are  appro¬ 
priate  for  multi-million  dollar  capital  investment  or  lease  programs. 

DoD  does  not  have  a  budget  line  item  or  budget  exhibit  imder  which  funding  for  DL 
can  be  identified;  thus,  there  is  no  formal  mechanism  to  tally  the  investments  made  in  any 
particular  DL  media.  Also,  it  is  easier  to  identify  hardware  and  circuit  costs,  since  those  are 
provided  by  third  parties,  whereas  courseware  development  and  conversion  may  be  accom¬ 
plished  using  internal  resources  which  are  harder  to  identify.  Installed  networks  have  been 
acquired  by  a  combination  of  leased  services  and  outright  purchases.  Systems  have  been 
acquired  in  ways  devised  to  match  available  funding— either  Operations  &  Maintenance  or 
Other  Procurement. 
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The  Services  have  the  responsibility  of  ultimately  directing  money  to  DL  programs. 
For  the  most  part,  the  Services  deserve  credit  for  installing  VTT  networks  out  of  the  limited 
funds  available.  However,  the  application  of  appropriated  funds  to  DL  is  inconsistent.  As 
an  example,  $17M  was  congressionally  mandated  for  Army  VTT  in  FY94.  The  Army  used 
$13M  to  cover  other  Army  priorities.  Of  the  remaining  $4M,  the  Army  spent  5  percent 
($200,000)  on  cotuseware  development,  25  percent  ($1M)  on  sustaining  VTT  overhead, 
and  70  percent  ($2.8M)  on  leasing  VTT  hardware  and  circuits.  Effectively,  only  the 
money  spent  on  the  courseware  development  contributed  to  establishing  the  academic 
infrastructure.  This  incident  is  not  isolated.  In  FY95,  $7.5M  was  appropriated  to  establish 
a  regional  demonstration  VTT  network  for  the  ARNG  (Ref.  16).  Several  constituencies 
have  emerged,  each  claiming  a  share  of  this  ^propriadon.  legislative  sponsors  want 
funds  to  be  directed  toward  purchasing  a  few  electronic  classrooms,  outfitting 
approximately  30,  and  eventually  leasing  the  aforementioned  to  local  community  interests. 
Of  the  sum  appropriated,  currently  less  than  10  percent  will  be  allocated  to  developing 
courseware  needed  by  the  ARNG  to  meet  readiness  requirements.  The  remaining  90  per¬ 
cent  has  been  identified  to  fund  engineering  services,  equipment  lease/purchase  acquisi¬ 
tions,  and  to  nuiintain  a  nonexistent  network  infrastructure.  By  not  funding  concurrent 
courseware  development,  the  VTT  systems  in  both  cases  will  be  severely  underutilized. 


IV.  DISCUSSION 


Evaluation  of  Service  DL  plans  has  generated  these  concerns:  (1)  too  little  use  of 
interactive  media;  (2)  not  enough  conversitm  to  DL;  (3)  selection  of  specific  VTT  tech¬ 
nologies  with  insufficient  review  of  requirements;  (4)  unceitain  cost  estimates  of  converting 
conventional  schoolhouse  courses  to  DL  courseware;  and  (5)  unknown  costs  and  benefits 
of  DL  in  a  Total  Force  context.  We  discuss  these  concerns  in  turn  below. 

A.  INTERACTIVE  MEDIA 

The  Army  and  the  Air  Force  plan  to  use  CBT  and  IVD  (or  CD)  for  48  percoit  and 
24  percent,  respectively,  of  DL  hours  for  their  RCs.  The  Navy  plan  would  use  these 
interactive  media  for  87  percent  of  the  DL  hours  it  proposes  for  RC  training.  However,  the 
87  percent  figure  is  misleading.  While  the  Army  and  the  Air  Force  plans  would  use 
interactive  media  for  over  40,0(X)  DL  hours  and  3,600  DL  hours,  respectively,  the  Navy’s 
plan  includes  less  than  1,0(K)  hours  of  interactive  media  for  its  RC  (see  Table  6).  This  is 
puzzling  since  empirical  evidence — much  of  it  firom  military  test  applications — suggests 
that  interactive  media  are  well  suited  for  delivering  complex,  technical  instruction.  The 
evidence  shows  better  training  effectiveness  (20  or  more  percent)  and  faster  learning 
(30  percent)  when  conventional  resident  courses  are  replaced  by  interactive  courses. 

B .  DL  CONVERSION 

The  Army  plans  to  convert  only  1 1  of  532  courses  to  more  than  80  percent  DL,  and 
only  7  of  those  532  to  100  percent  DL.  Thus,  even  with  a  DL  system  in  place,  travel  by 
RC  students  to  the  resident  training  schoolhouse  still  would  be  required  for  525  courses.  If 
DL  could  be  substituted  for  course  segments  that  die  Army  currently  teaches  in  residence, 
the  travel  and  per  diem  savings  would  be  substantial,  using  the  ARNG  planning  figure  of 
$800  per  student  for  travel  cost  for  resident  training.  In  FY94,  about  75,0(X)  enlisted 
personnel  in  the  Army  RCs  attended  resident  training  courses.  If  one-half  to  two-thirds  of 
the  75,000  students  were  able  to  take  DL  courses,  which  deliver  cognitive  training  equiv¬ 
alent  to  that  delivered  in  the  schoolhouse,  the  travel  savings  in  FY94  would  have  been 
between  $30M  and  $40M. 


Many  DL  technologies  simulate  experience  with  real  equipment  through  the  use  of 
video  displays  or  similar  means.  Two-dimensional  (2D)  representations  of  systems  or 
components  can  be  transmitted  by  DL  tedinologies  [which  can  also  present  three- 
dimensional  (3D)  perspectives  of  these  items]  to  support  “almost  hands-on”  training  that,  in 
many  applications,  is  as  effective  as  3D  hands-on  training — and  less  costly.  Systematic 
evaluations  of  3D  and  2D  maintenance  simulators  have  found  few  differences  in  test 
performance  of  students  who  were  trained  on  real  equipment  and  students  trained  on 
simulated  equipment. 

It  is  also  notable  that  medical  schools  use  synthetic  models  of  hearts,  eyes,  and 
brains  to  provide  surgeons  the  psychomotor  training  and  practice  required  to  work  on  these 
and  other  human  organs.  If  such  mock-ups  are  effective  for  hands-on  training  of  those 
who  repair  die  human  anatomy,  the  Services  may  find  equal  effectiveness — and  cost 
savings — by  developing  and  distributing  to  DL  centers  3D  models  to  train  military 
personnel  to  repair  weapons,  vehicles,  and  other  equipment. 

The  professional  developmait  and  leadership  courses  for  the  Army  RCs  arc  also 
good  prospects  for  100  percent  conversion  to  DL  media.  A  reason  often  cited  for  having 
some  part  of  a  course  delivered  in  residence  is  the  occurrence  of  formal  or  informal  teaming 
that  facilitates  learning  as  students  help  each  other.  Such  facilitation  might  be  achieved  by 
teaming  students  in  die  same  course  at  a  DL  center.  If  the  students  in  a  DL  class  are  not 
collocated,  they  can  be  provided  with  class  rosters,  personnel  vitae,  telephone  numbers, 
and  e-mail  addresses  to  facilitate  their  interactions.  Student  custody  of  DoD-owned 
personal  computers  and  modems  for  use  at  home  during  DL  courses  is  a  possible 
arrangement  for  Internet  communications. 

C.  VTT  NETWORKS 

The  Navy’s  choice  of  a  2V,  2A  network  for  CESN  has  proven  workable  for  a  wide 
range  of  Navy  courses  at  select  locations,  especially  since  CESN  is  involved  in  courseware 
as  well  as  network  operations  and  can  therefore  document  usage.  However,  in  pursuing  its 
2V,  2A  architecture,  the  NR  has  advanced  a  VTT  DL  plan  that  cannot  effectively  leverage 
CESN’s  education  and  training  resources  infrastracture.  At  tqtproximately  $75,(X)0  per  site 
per  year  for  CESN,  funding  is  a  major  consideration.  At  that  cost,  installing  197  M.. 
locations  would  be  impracticable  for  the  NR.  The  NR  has  proposed  an  outyear  $5-$7M 
investment  in  a  desktop  video  teleconferencing  equipment  suite  using  a  reduced  circuit 
edacity.  While  the  circuits  can  connect  with  CESN,  data  transmission  rate  would  default 


to  the  NR’s  lowest  circuit  speed  and  thereby  compromise  die  quality  of  instruction  for  all 
sites  involved  in  the  DL  session. 

The  Army’s  choice  of  a  2V,  2A  network  for  TNET  relies  on  a  more  restrictive 
communications  architecture  than  the  Navy’s  CESN.  Although  both  use  die  same  basic 
hardware  in  the  classroom,  TNET  is  more  expensive  to  lease.  TNET  is  intended  to  deliver 
only  hardware  and  communications  services  to  trainers  and  users.  Unlike  CESN,  TNET 
does  not  manage  the  courseware  component  of  this  DL  triplication.  The  ARNG  is 
exploring  a  scaled-down  2V,  2A  VTT  application  using  commercial  VTC  equipment.  As 
proposed,  this  application  would  not  function  with  the  existing  TNET  or  CESN  networks. 

The  Air  Force  Reserve  (AFRES)  leases  47  TNET  sites  and  obtains  network 
management  services  from  the  Army.  It  is  now  possible  to  interconnea  the  AFRES  sites 
with  the  ATN  and  Warrior  Networks,  by  defaulting  their  2V,  2A  ardiitecture  to  a  IV,  2A 
medium.  The  default  to  IV,  2A  undermines  die  TNET  contention  that  all  sites  must  have 
the  ability  to  transmit  video. 

Each  RC  that  plans  to  use  or  is  using  a  2V,  2A  network  defends  its  choice  by  citing 
the  ability  of  2V,  2A  to  conduct  VTC.  Identifying  VTC  applications  serves  to  undermine 
the  business  case  for  VTT  unless  VTC  users  are  contributing  resources  (funds,  personnel, 
equipment).  Classroom  space  can  be  used  as  conference  rooms,  and  vice  versa,  but  diey 
are  less  effective  when  not  used  as  configured.  Only  CESN  can  (historically)  defend  its 
VTC  dual-use  position  by  being  able  to  o^t  costs  against  the  DL  mission  alone.  Draft 
DoD  policy  does  not  state  a  preference  (one-  or  two-way)  VTT  but  does  stress  the  value  of 
interoperability  with  other  ardiitectuies  and  of  the  ability  to  gateway  among  networks. 

D.  CONVERSION  COST 

The  range  of  estimates  of  costs — $526M  to  $1,833M — ^to  convert  conventional 
schoolhouse  courses  to  DL  is  very  wide.  But  even  the  low-end  cost  is  a  formidable 
obstacle  to  irnplementing  DL  for  the  RCs  alone.  This  is  a  major  undertaking  that  will 
require  several  years  to  implement. 

E.  COST-BENEFIT  ANALYSIS 

Adoption  of  DL  for  the  Service  RCs  requires  a  large  investment  in  course  conver¬ 
sion.  Conversion  requires  an  investment  up  front  while  resident  training  programs 
continue;  payback  in  the  form  of  travel  and  per  diem  cost  savings  begins  as  DL  comes  on 
line  and  replaces  resident  courses.  Such  an  investment  is  difficult  to  defend  without 


considering  the  costs  and  benefits  of  applying  DL  to  the  Total  Force.  The  demonstrated 
effectiveness  of  DL  should  provide  advantages  to  the  ACs  as  well  as  to  the  RCs. 
Therefore,  the  Services  cannot  be  expected  to  make  adequate  fiscal  plans  for  DL  until  costs 
and  benefits  in  a  Total  Force  context  are  evaluated.  A  number  of  issues  should  be  resolved; 
Which  courses,  of  those  already  identified  for  conversion  to  DL  for  the  RCs,  would  be 
useful  to  the  ACs?  What  additional  course  conversions  would  be  needed?  How  much 
would  that  cost?  What  effect  would  inclusion  of  AC  users  have  on  the  cost  of  DL 
netwoiks? 

Potential  gains  for  the  Services  in  training  effectiveness  should  be  determined  by 
trading  off  AC  resident  training  for  DL.  Other  benefits,  some  of  which  are  not  easy  to 
quantify  without  examining  DL  in  a  Total  Force  context,  are  the  following: 

•  DL  will  save  travel  and  per  diem  costs  for  resident  training.  Travel  and  per 
diem  for  the  Total  Force  will  exceed  by  an  unknown  amount  the  $55M  per 
year  for  the  RCs  alone. 

•  The  conversion  of  more  resident  courses  to  100  percent  DL  will  save  addi¬ 
tional  RC  travel  funds. 

•  Interactive  courseware  is  often  more  effective  and  more  efficient  than  conven¬ 
tional  classroom  instruction. 

•  Adoption  of  DL  should  permit  a  planned  reduction  of  the  infrastructure  that 
supports  only  resident  training. 

•  There  will  be  a  reduction  in  course  conversion  costs  by  eliminating  as  yet 
unidentified  duplications  among  courses  that  the  Services  nominated  for 
conversion. 

•  Configuration  standards  and  joint-Service  courseware  will  avoid  duplications 
that  exist  today  in  the  Services’  resident  training  curricula. 

•  Development  of  joint-Service  courseware  and  the  shared  usage  and  cost  of 
networks  for  both  the  ACs  and  RCs  should  stimulate  each  Service  to  pay  for 
only  what  is  necessary. 


V.  FINDINGS 


The  following  findings  are  derived  from  (1)  evaluation  of  Service  plans  for  using 
DL  technologies  to  train  RC  personnel  and  (2)  the  continuing  informal  review  of  Service 
training  requirements  and  training  technology. 

A.  POTENTIAL  OF  DL 

•  Die  efficacy  of  DL  technologies,  which  oiable  learning  to  take  place  without 
an  instnictor  being  physically  present,  has  been  demonstrated  in  the  military, 
academia,  and  the  commercial  world  by  more  than  300  studies.  DL  is  as  effec¬ 
tive  as  conventional  resident  instruction  in  achieving  a  wide  range  of  instruc¬ 
tional  objectives  (Section  in.  A  and  the  Appendices). 

•  Many  cost-effective  applications  of  DL  technologies — ^i.e.,  computer-based 
training,  interactive  videodisc  training,  compact  disc  training,  and  video  tele¬ 
training — have  involved  AC  personnel  and  DoD  civilians  in  schoolhouse 
courses  (Section  IILA).  Thus,  besides  delivering  DL  to  the  RCs,  these 
technologies  could  be  used  to  train  AC  personnel  and  DoD  civilians  either  at  a 
distance  or  in  residence. 

•  DL  can  be  used  to  train  those  who,  for  whatever  reason — usually  a  lack  of 
funds — are  unable  to  travel  to  resident  training  sites.  DL  could  have  trained 
the  nearly  46,(XX)  Army  and  Navy  RC  personnel  who  required  trainiiig  in 
FY95,  in  contrast  to  the  approximately  20,000  RC  personnel  who  were 
expected  to  receive  resident  training  in  that  year  (Section  n.  Table  3). 

•  Fiscal  savings  in  travel  and  per  diem  (which  are  relatively  easy  to  estimate), 
and  reductions  in  time  to  train,  training  material  costs,  instmctor  costs,  and 
materials  preparation  and  modification  costs  are  potentially  available  from  use 
of  DL  technologies  (Section  n.B). 

•  Increased  readiness  obtainable  from  comprehensive  use  of  DL  technologies  is 
difficult  to  estimate,  but  it  may  prove  to  be  the  primary  payoff  from  systematic 
adoption  of  DL  technologies  for  RC  training  (Section  n.B). 


B .  DL  IN  DoD  TODAY 


•  Investment  reported  by  the  Services  in  DL  technologies  is  limited  and  their  use 
is  rare — overall  use  is  estimated  to  be  less  than  1  percent  of  Service  RC 
training  activity  (Section  in.E). 

•  Most  Service  expenditures  for  DL  are  in  hardware  rather  than  courseware,  a 
practice  that  limits  the  utilization  of  DL  systems  (Section  in,F). 

C.  UP-FRONT  INVESTMENT 

•  Conversion  to  DL  courseware  requires  an  up-firont  investment  while  resident 
training  programs  continue;  financial  payback  in  the  form  of  travel,  per  diem, 
and  schoolhouse  cost  savings  begins  as  DL  comes  on  line  and  resident  courses 
are  phased  out  or  reduced  (Section  IV.E), 

•  Estimates  of  costs  to  convert  conventional  schoolhouse  instruction  to  M- 
courseware  vary  widely.  Service  investment  cost  estimates  for  RC  DL  are 
sizable,  varying  from  about  $500M  to  $1,800M.  However,  eliminating  dupli¬ 
cation  among  the  1,097  courses  that  the  Services  independently  propose  to 
convert  to  DL  would  reduce  these  conversion  costs  (Section  in.B). 

D.  DL  CONVERSION  PLANS 

•  The  Services  plan  to  convert  about  19  percent  of  their  courses  (1,097  of 
5,847)  for  RC  training  to  DL.  Although  the  Navy  plans  to  convert  only  one 
percent  of  its  courses  (27  of  2,433)  for  RC  training,  the  Navy  already  has 
about  600  DL  courses  in  a  print  format  (Section  II). 

•  While  the  Navy  and  Air  Force  plans  indicate  100  percent  conversion  of  all 
prospective  DL  courses,  the  Army  plan  calls  for  100  percent  DL  conversitm  of 
only  7  of  532  courses  suitable  for  conversion  (Section  H,  Table  4).  Thus, 
most  Army  RC  personnel  in  DL  courses  would  still  travel  to  resident  training 
sites. 

E.  JOINT  SERVICE  DL 

•  Configuration  standards  are  essential  to  ensure  DL  interoperability  among  the 
Services  and  among  elements  within  eadi  Service  (Section  in.D). 

•  Joint  management  of  DL  courseware  development  is  needed  to  avoid  duplica¬ 
tion  of  course  material;  Joint-Service  courseware  can  be  used  in  many  cases  in 
which  Service  courses — ^whole  or  in  part — are  the  same  or  similar  to  those  of 
another  Service  (Section  m.D). 
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F.  DATA  GAPS  AND  DL  INVESTMENT  DECISIONS 


•  Data  furnished  by  die  Services  differ  widely  in  quality  and  comprehensive¬ 
ness.  Despite  some  commendable  efforts  to  respond  conscientiously  to  the 
DepSecDef  tasking,  gaps  and  uncertainties  remain  in  all  Service  plans  for 
using  DL  in  RC  training  (Sections  n,  IV). 

•  Service  budgets  provide  ho  budget  line  items  or  budget  exhibits  under  which 
funding  for  DL  can  be  uniformly  identified,  tracked,  and  managed  (Section  II). 

•  Service  plans  do  not  identify  the  marginal  cost  of  the  schoolhouse  infra¬ 
structure  (instructors,  training  developers,  training  equipment,  course  main¬ 
tenance,  school  operations,  etc.)  or  discuss  how  implementing  DL  would 
affect  the  present  schoolhouse  infrastructure  and  costs  (Section  II.C). 

•  Opportunities  to  use  DL  technologies  to  train  1.6  million  personnel  in  the  ACs 
and  900,(XX)  DoD  civilians,  either  in  residence  or  at  a  distance,  have  not  been 
compiled  or  studied  (Section  in.C). 

•  Effective  strategic  plans  for  RC  training  cannot  be  devised  without  full  con¬ 
sideration  of  Total  Force  training  strategies  and  plans  (Section  IV.E).  Witii 
few  exceptions,  the  content  of  courses  is  the  same  for  AC  and  RC  personnel  in 
the  same  job  occupations.  Thus,  DL  Courseware  could  be  used  by  the  Total 
Force,  whose  larger  population  makes  the  economics  of  using  DL  technologies 
more  attractive. 


VI.  RECOMMENDATIONS 


Based  on  the  above  findings,  senior  representatives  from  the  Office  of  the  Secretary 
of  Defense,  the  Joint  Staff,  the  four  Military  Services,  and  other  Defense  Agencies  should 

1.  Examine  DL  in  a  Total  Force  context:  all  Services,  all  components,  and  DoD 
civilians.  This  should  include  schoolhouse  infrastructure  cost,  DL  infra¬ 
structure  cost,  travel  and  per  diem  costs,  costs  associated  with  resident 
training,  and  a  thorough  review  of  the  costs  of  converting  conventional 
classroom  courses  to  DL  technologies,  and  expected  improvement  in  both 
training  efficiency  and  personnel  readiness. 

2.  Prepare  a  cohesive,  Defense-wide  action  plan  for  DL  that  fully  capitalizes  on 
the  opportunities  provided  by  DL  technologies  and,  if  anticipated  improve¬ 
ments  in  training  efficiency  and  personnel  readiness  warrant,  identify  mile¬ 
stones  that  expeditiously  move  the  plan  forward  for  implementation.  This 
action  plan  should  address  the  need  for  developing  a  DoD  organizational 
structure  to  assist  in: 

•  Guiding  and  assessing  the  application  of  DoD  DL  technologies  in  a  Total 
Force  context  that  includes  ACs,  RCs,  and  the  Defense  civilian  work¬ 
force; 

•  Ensuring  that  DoD  DL  technologies  are  develojied,  managed,  and  used  in 
a  cost-effective  manner;  and, 

•  Identifying  and  enforcing  configuration  standards  for  DoD  DL  tech¬ 
nologies. 
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GLOSSARY 


IV,  2A 

one-way  video,  two-way  audio 

2D 

two  dimensional 

2V,  2A 

two-way  video,  two-way  audio 

3D 

three  dimensional 

AC 

active  component 

AFRES 

Air  Force  Reserve 

ANG 

Air  National  Guard 

ARNG 

Army  National  Guard 

ATN 

Air  Technology  Network 

ATRRS 

Army  Training  Requirements  and  Resources  System 

B 

billion 

CAI 

computer-aided  instruction 

CBT 

computer-based  training 

CD 

compact  disc 

CD-ROM 

CD-read-only  memory 

CDV 

compressed  digital  video 

CESN 

CNET  Electronic  Schoolhouse  Network 

CIF 

common  intermediate  format 

CMI 

computer-managed  instruction 

CNET 

Chief  of  Naval  Education  and  Training 

CPH 

cost  per  hour 

DepSecDef 

Deputy  Secretary  of  Defense 

DISA 

Defense  Information  Systems  Agency 

DL 

Dist^ce  Learning 

DoD 

Department  of  Defense 

FTS 

Federal  Telecommunications  System 

FY 

fiscal  year 

GSA 

General  Services  Administration 

ICW 

interactive  courseware 

ISDN 

Integrated  Services  Digital  Network 

IVD 

Kbps 

L 

M 

M&RA 

Mn^STD 

MOS 

MOSQ 

MRAI&RE 

NCO 

NQ 

NR 

MTU 

OSD 

P 

PDM 

PMEC 

POM 

Q 

RC 

ROM 

SEP 

SEW 

TOY 

TNET 

TRADOC 

VT 

VTC 

VTT 

WN 


interactive  videodisc 
kilobits  per  second 
lease 
million 

Manpower,  Reserve  Affairs 
Military  Standard 
military  occupational  specialty 
MOS  qualified 

Manpower,  Reserve  Affairs,  Installations,  and  Environment 
non-commissioned  officer 
not  qualified 
Naval  Reserve 

National  Technological  University 
Office  of  the  Secretary  of  Defense 
purchase 

Program  Decision  Memorandum 
Professional  Military  Education  Center 
Program  Objective  Memorandum 
qualified 

reserve  component 
read  only  memory 
Satellite  Education  Program 
Satellite  Education  Network 
temporary  duty 
Teletraining  Network 
Training  and  Doctrine  Command 
video  tape 

video  teleconferencing 
video  teletraining 
Warrior  Network 
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APPENDIX  A 
CAI  EVALUATION 


Orlansky  and  String  evaluated  the  effectiveness  and  cost  of  computer-assisted 
instruction  (CAI)  and  computer-managed  instruction  (CMI)  for  military  training  on  the 
basis  of  30  studies  involving  48  datasets  and  the  17  courses  indicated  in  Table  A-1.*  In 
their  evaluation,  they  compared  four  methods  of  instruction: 

Conventional  Instruction:  group-paced  lectures,  and  discussions. 

Individualized  Instruction:  self-paced  without  computer  support. 

Computer-Assisted  Instruction:  computer  stores  and  provides  instruc¬ 
tion^  materials  to  students  individually  via  interactive  terminals;  computer 
tests  and  guides  students;  self-paced. 

Computer-Managed  Instruction:  instructional  materials  and  tests  provided 
away  from  computer;  computer  scores  and  tests  and  guides  students;  self- 
paced. 

Most  (70  percent)  of  the  data  on  CAI  came  from  experiments  with  few 
students  (up  to  50)  and  limited  course  materials  (1  day  to  1  week).  There 
are  fewer  studies  of  CMI  but  these  involved  more  students  (600  to  2,500) 
and  longer  courses  (2  to  10  months).  [As  shown  by  Table  A-1,]  there  is  a 
wide  range  of  subject  matter  in  these  studies,  e.g.,  knowledge,  theory,  and 
hands-on  performance  skills;  electronics  machinist,  recipe  conversion, 
vehicle  repair,  fire-control  technician. 

Each  of  the  30  studies  reported  effectiveness.  However,  only  eight  of  the 
studies  that  report  effectiveness  also  provide  some  cost  data.  The  latter  data 
are  limited  to  expenses  incurred  during  the  experiment  and  are  incomplete 
with  respect  to  costs  of  program  management,  maintenance  and  repair, 
instructional  support,  and  other  factors  important  in  determining  life-cycle 
costs. 

There  are  two  ways  of  evaluating  the  cost-effectiveness  of  alternative 
military  systems.  Given  two  systems  of  the  same  cost,  one  would  prefer 
the  system  that  provides  greater  effectiveness.  Given  two  systems  of  the 
same  level  of  effectiveness,  one  would  prefer  the  system  that  costs  less. 


*  Orlansky,  Jesse,  and  String,  Joseph,  Cost-Effectiveness  of  Computer-Based  Instruction  in  Military 
Training,  IDA  Paper  P-1375,  April  1979. 
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Table  A*1.  Courses  Used  In  Various  Studies  of  CAI  and  CMI 


Courses 

No.  of  Evaluations 

CAI 

CMI 

Basic  Electronics 

15 

Electricity 

5 

Machinist 

2 

Training  Materials  Development 

1 

Recipe  Conversion 

2 

Aircraft  Panel  Operation 

1 

Medical  Assistant 

4 

Vehicle  Repair 

4 

Weather 

1 

Tactical  Coordinator  (S-3A) 

1 

Fire  Control  Technician 

4 

Aviation  Familiarization 

2 

Aviation  Mechanical  Fundamentals 

2 

Inventory  Management 

1 

Materiel  Facilities 

1 

Preciston  Measuring  Equipment 

1 

Weapons  Mechanic 

1 

Total 

40 

8 

All  [30]  studies  of  computer-assisted  and  computer-manag^  instruction 
have  used  the  second  approach.  Computer-based  instructional  systems 
have  been  design^  to  provide  the  same  degree  of  effectiveness  (student 
achievement)  as  the  method  of  instruction  they  might  replace  (conventional 
instruction).  Therefore,  these  alternative  methods  of  instruction  must  be 
evaluated  in  terms  of  differences  in  their  costs. 

The  effectiveness  of  computer-assisted  and  computer-managed  instruction 
for  military  training  has  been  measured  only  by  student  achievement  at 
school  and  not  by  performance  on  the  job.  Correlations  between  perform¬ 
ance  at  school  and  on  the  job  have  not  been  established  for  any  method  of 
instruction.  Data  on  length  of  time  required  for  students  to  complete  a 
course  (generally  less  for  CAI  and  CMI  than  for  conventional  insdiiction) 
should  be  treated  as  a  measure  of  the  cost  of  instruction  rather  than  a 
measure  of  its  effectiveness.  The  same  argument  applies  to  academic 
attrition  rate.  The  attitudes  of  students  and  instructors  to  CAI  ^d  CMI  may 
be  interesting;  however,  they  are  qualitative  in  nature  and  it  is  difficult  to 
relate  such  data  either  to  the  cost  or  the  effectiveness  of  instruction. 


In  the  30  studies,  CAI  or  CMI  was  generally  compared  to  conventional  instruction; 
the  time  savings  found  in  these  comparisons  may  be  due  to  a  combination  of  self-pacing, 
computer  support,  and  revised  and  possibly  reduced  course  material. 

CONCLUSIONS 

Student  Achievement.  Student  achievement  in  courses  at  military 
training  schools  witii  computer-assisted  instruction  is  the  sarrc  as  or  greater 
than  that  with  conventional  instruction;  the  amount  of  additional  achieve¬ 
ment  is  small  and  has  little  practical  importance.  Student  achievement  in 
course  with  computCT-managed  instruction  is  about  the  same  as  that  with 
conventional  instruction.  Both  of  these  results  are  due  to  keeping  students 
in  CAI  and  CMI  courses  until  they  achieve  standards  set  previously  for 
conventional  instruction. 

Student  Time  Saving.  Computer-assisted  and  computer-managed 
instruction  in  military  traiiting  save  atout  30  percent  of  the  time  (median 
value)  needed  by  students  to  complete  the  same  courses  given  by  conven¬ 
tional  instruction.  The  amounts  of  time  reported  as  saved  vary  widely,  but 
little  attention  has  been  given  to  the  factors  that  could  account  for  the  wide 
variation.  Most  of  the  results  on  computer-assisted  instruction  come  tom 
experiments  of  limited  duration,  with  limited  amounts  of  course  materials, 
and  with  relatively  few  students.  Where  computer-managed  instruction  has 
been  used  for  extended  periods  (up  to  4  years),  the  initial  time  savings  have 
been  maintained  or  increased. 

The  above  conclusions  are  shown  in  summary  form  in  Table  A-2  along  with 
findings  related  to  other  measures.  The  sources  of  the  studies  evaluated  by  Orlansky  and 
String  are  identified  in  Table  A-3. 


Table  A-2.  Summary  of  Findings  on  CAI  and  CMI, 
Compared  to  Conventional  Instruction 


Measure 

Finding(Compared  to 
Conventional  Instruction) 

Comments 

CAI 

CMI 

Student  Achievement 

Same  or 
more 

Same 

Performance  measured  only  at 
school. 

Relation  between  performance 
at  school  and  on  the  job  not 
measured. 

Obsen/ed  differences  not  of 
practical  importance. 

No.  of  Comparisons 

40 

8 

CMI:  Most  time  savings  main¬ 
tained  or  increased  with 
extended  use. 

Time  saved  (Median) 

29% 

44% 

Range 

-31  to  89% 

12  to  69% 

No.  of  Comparisons 

5 

7 

Computer  support  saves  little  time 
beyond  that  of  individualized 
instruction. 

Time  saved 

•  Individualized  Instruction 

•  CAI.  CMI 

64% 

69% 

51% 

Student  Attrition 

About  the 
same 

Slight  increase 
may  occur 

CAI:  Very  limited  data. 

CMI:  Possible  decline  in  student 
quality. 

Student  Attitudes 

Favorable 

Favorable 

Instructor  Attitudes 

Unfavorable 

Unfavorable 

Very  limited  data. 

Little  attention  given  to  instructors. 

Cost 

Less,  due  to 
student  time 
savings 

Less,  due  to 
student  time 
savings 

Data  limited  and  incomplete. 

Cost-effectiveness 

Not  known  because  cost  data  are 
limited  and  incomplete. 
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Fletcher  used  a  “meta-analytic”  technique  to  review  the  effectiveness  and  cost  of 
interactive  videodisc  (IVD)  instruction.*  The  review  involved  47  comparisons  derived 
from  28  studies  of  the  effectiveness  of  IVD  applications  in  training  and  education. 
Table  B-1  identifies  sources  of  the  studies.  (More  than  28  sources  are  referenced 
inasmuch  as  some  studies  were  reported  by  two  sources.)  The  baseline  for  comparing  IVD 
instruction  with  conventional  instruction  involved  three  instructional  settings:  mihtary 
training,  industrial  training,  and  higher  education. 

The  meta-analytic  approach,  which  is  an  analysis  of  analyses,  uses  (1)  a  quantita¬ 
tive  measure  called  “effect  size”  to  tabulate  the  outcomes  of  collected  studies  including 
those  whose  results  are  not  statistically  significant  and  (2)  statistical  procedures  to  syn¬ 
thesize  the  quantitative  measures  and  describe  the  findings.  The  studies  are  made 
comparable  by  converting  them  to  an  effect  size  (ES)  metric,  which  is  defined  as  the 
standardized  mean  difference  in  test  scores  of  experimental  (e)  and  control  (c)  groups. 
Thus,  ES  =  (xe  “  Xc)/Sc .  where  x  is  the  mean  test  score  and  S  is  the  standard  deviation. 

FINDINGS 

Fletcher’s  findings,  which  follow,  concern  the  instructional  capabilities  provided  by 
rVD  systems,  not  the  hardware  itself. 

1 .  Effectiveness 

Interactive  videodisc  instruction  was  used  successfully  to  teach.  Four 
comparisons  of  interactive  videodisc  instraction  with  a  “placebo”  treatment,  in  which  no 
relevant  instraction  was  given,  resulted  in  an  average  increase  in  achievement  of  1.39 
standard  deviations,  or,  roughly,  an  increase  in  the  achievement  for  50th  percentile  students 
to  about  the  92nd  percentile  of  achievement. 


•  Fletcher,  J.D.,  Effectiveness  and  Cost  of  Interactive  Videodisc  Instruction  in  Defense  Training  and  Edu¬ 

cation,  IDA  Paper  P-2372,  July  1990. 


OveraiU  interactive  videodisc  instruction  was  more  effective  than 
conventional  instruction.  Forty-seven  comparisons  of  interactive  videodisc  instruction 
with  ctMiventional  approaches  were  identified  for  this  review.  Over  all  instructional  settings 
(military  training,  industrial  training,  higher  education),  instructional  approaches  (simu¬ 
lation  only,  tuttwial  only,  simulation  and  tutorial  combined),  and  outcomes  (knowledge, 
performance,  retention),  interactive  videodisc  instruction  increased  achievement  an  average 
of  0.50  standard  deviations  over  conventional  instruction  (an  increase  for  50th  percentile 
students  to  about  the  69th  percentile  of  achievement). 

Interactive  videodisc  instruction  was  more  effective  than  conven¬ 
tional  instruction  in  military  training.  Twenty-four  comparisons  of  interactive 
videodisc  instruction  with  conventional  approaches  to  mihtary  maintenance,  operator,  and 
command  training  were  identified  for  this  review.  Over  all  instructional  approaches  and 
outcomes,  they  showed  an  average  increase  in  achievement  of  0.38  standard  deviations  (an 
increase  for  50th  percentile  students  to  about  the  65th  percentile  of  achievement). 

Interactive  videodisc  instruction  was  more  effective  than  conven¬ 
tional  instruction  in  higher  education.  Fourteen  comparisons  of  interactive  video¬ 
disc  instruction  with  conventional  approaches  to  instruction  in  higher  education  settings 
(colleges  and  universities)  were  identified  for  this  review.  Over  all  instructional  approaches 
and  outcomes,  they  showed  an  average  increase  in  achievement  calculated  using  pooled 
standard  deviations  of  0.69  standard  deviations  (an  increase  for  50th  percentile  students  to 
about  the  75th  percentile  of  achievement).  The  lower  average  effect  sizes  found  for  military 
training  than  for  higher  education  may  be  due  to  a  focus  in  mihtary  training  on  reaching 
threshold  level(s)  of  achievement  with  minimized  costs  and  time  so  that  students  who  reach 
achievement  criteria  are  sent  on  to  duty  assignments  rather  than  held  in  the  instractional 
setting  as  they  are  in  higher  education. 

Interactive  videodisc  instruction  was  equally  effective  for  both 
knowledge  and  performance  outcomes.  The  average  effect  sizes  calculated  using 
pooled  standard  deviations  for  27  knowledge  outcomes  (facts,  concepts,  and  other  infor¬ 
mation  students  acquired)  and  for  20  performance  outcomes  (procedures,  skills,  and  other 
capabilities  students  could  demonstrate)  both  averaged  around  0.35  standard  deviations 
(suggesting  an  improvement  of  50th  percentile  students  to  about  the  64th  percentile). 

The  more  the  interactive  features  of  interactive  videodisc  technology 
were  used,  the  more  effective  the  resulting  instruction.  One  study  examined 
this  issue  directly  and  found  greater  achievement  with  greater  levels  of  interactivity.  The 


issue  can  also  be  examined  by  comparing  the  effect  sizes  for  Level  II  videodisc  instruction 
(-.70,  overall)  with  those  for  Level  III  videodisc  instruction  (.50,  overall). 

Directed,  tutorial  approaches  were  more  effective  than  stand-alone 
simulations  in  interactive  videodisc  instruction.  Effect  sizes  averaged  0.70  for 
tutorial  approaches,  0.40  for  combined  tutorial  and  simulation  approaches,  and  0.15  for 
simulations  by  themselves. 

Within-group  variability  was  smaller  in  interactive  videodisc  instruc¬ 
tion  than  in  conventional  instruction.  In  nearly  every  comparison  of  interactive 
videodisc  instraction  with  conventional  instruction,  achievement  in  the  interactive  videodisc 
instruction  group  was  less  variable  than  in  the  conventional  instruction  group,  suggesting 
there  was  more  equitable  distribution  of  achievonent  using  interactive  videodisc 
instruction. 

Interactive  videodisc  instruction  was  more  effective  than  computer- 
based  instruction  without  videodisc  interaction.  Both  interactive  videodisc 
instruction  and  computer-based  instmction  have  been  found  to  be  more  effective  than 
conventional  instruction.  However,  the  average  effect  size  of  around  0.69  observed  for 
interactive  videodisc  instruction  used  in  colleges  is  considerably  higher  than  both  the 
average  effect  size  of  0.26  found  in  a  review  of  computer-based  instruction  used  in  colleges 
and  the  average  effect  size  of  0.42  found  in  a  review  of  computer-based  instruction  used  in 
adult  education. 

There  was  little  in  the  reviewed  studies  to  indicate  how  interactive 
videodisc  instruction  achieves  its  success.  The  studies  examined  in  this  review 
did  little  to  indicate  which  features  of  interactive  videodisc  instruction  contribute  to  the 
observed  increases  in  student  achievement.  In  addition,  there  are  many  outcomes — such  as 
speed  of  response,  accuracy  of  response,  attitude  toward  the  subject  matter,  insight, 
transfer,  and  retention — ^that  alone  or  in  some  combination  may  become  the  objective(s)  of 
instruction.  How  different  designs  for  intCTactive  videodisc  instruction  contribute  to 
accomplishing  these  different  outcomes  was  rarely  addressed  by  the  reviewed  studies  and 
remains  a  proper  topic  for  future  research. 

2 .  Cost 

Interactive  videodisc  instruction  was  less  costly  than  conventional 
instruction.  All  13  cost  ratios  (the  ratio  of  costs  for  interactive  videodisc  instruction  over 
costs  for  conventional  instmction)  found  for  this  review  were  less  than  1.0,  indicating 
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lower  costs  in  every  measured  instance  for  interactive  videodisc  instruction.  The  average 
across  all  13  cost  ratios  was  0.36.  The  average  amount  of  student  time  saved  across  the 
studies  covered  by  this  review  was  31  percent,  suggesting  another  source  of  savings  from 
use  of  interactive  videodisc  instruction.  Reductions  in  total  mihtary  manpower  that  can 
result  from  such  time  savings  in  training  may  be  an  additional,  significant  source  of  cost 
savings,  but  this  possibility  was  not  examined  here. 

3 .  Cost-Effectiveness 

Interactive  videodisc  instruction  was  more  cost-effective  than  con¬ 
ventional  instruction.  Interactive  videodisc  instruction  was  found  to  be  both  less  costly 
in  all  studies  that  reported  costs  and  more  effective  overall  than  the  conventional  approaches 
with  which  it  was  compared.  This  finding  should  be  viewed  as  suggestive  rather  than 
conclusive,  since  no  studies  were  found  that  used  systematic  models  to  provide  empirical 
data  on  both  cost  inputs  and  effectiveness  outputs. 

4 .  Time  on  Task 

Interactive  videodisc  instruction  may  increase  time  on  task.  Only  one 
study  reported  an  observation  relative  to  this  point.  This  study  reported  a  45  percent 
increase  in  the  time  spent  practicing  a  targeted  task  as  a  result  of  the  introduction  of 
interactive  videodisc  instruction.  This  result  was  viewed  as  promising,  but  not  conclusive. 


5 .  Retention 

Interactive  videodisc  instruction  seems  unlikely  to  affect  retention. 

Four  studies  addressed  retention.  Both  negative  and  positive  results  moved  closer  to  zero 
over  the  retention  intervals  considered  in  these  studies.  Post-training  experiences  typically 
influence  retention  of  knowledge  and  performance  in  a  powerful  manner.  Findings  for 
interactive  videodisc  training  corroborate  other  Defense  experiences  with  new  training 
approaches  that  suggest  their  value  lies  primarily  in  improving  the  efficiency  of  instruction, 
not  in  assuring  longer  retention. 

CONCLUSION 

The  Fletcher  meta-analysis  showed  interactive  videodisc  instruction  is  both  more 
effective  and  less  costly  than  conventional  instruction.  The  47  comparisons  suggest, 
therefore,  that  interactive  videodisc  instruction  can  have  a  significant  positive  impact  on 
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military  training  and  education.  Although  more  needs  to  be  learned  about  how  interactive 
videodisc  instruction  should  be  designed  and  employed,  it  should  now  be  routinely 
considered  and  used  in  military  training  and  education. 
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In  a  study  of  issues  related  to  distance  training  technologies.  Redding  and  Fletcher 
gathered  data  firom  many  different  evaluations  to  get  an  overall  view  of  the  effectiveness  of 
interactive  courseware  (ICW).*  In  addition  to  considering  Fletcher’s  findings  on 
interactive  videodisc  (IVD),  which  appear  in  Appendix  B,  Redding  and  Fletcher  presented 
results  of  numerous  studies  of  computer-based  instruction  (CBI).  These  studies  used  the 
meta-analytic  methodology  described  in  Appendix  B.  The  measure  of  effectiveness  is 
“effect  size,”  which  is  the  difference  between  the  test-score  means  (of  an  expoimental 
group  and  a  control  group)  divided  by  the  standard  deviation  of  the  control  group  or  by  the 
standard  deviation  of  the  pool  of  the  control  and  experimental  groups.  Table  C-1 
summarizes  effect  sizes  for  almost  200  studies  of  CBI  in  various  settings. 


Table  C-1.  Effect  Sizes  for  CBI 


Setting 

Effect 

Size 

Number 

of 

Studies 

Percentile 
Improvement  in 
Achievement® 

Researchers'* 

Elementary  School 

.47 

28 

50^68 

Kulik,  C-L  Kulik, 
Bangert-Drowns 

Secondary  School 

.40 

42 

50^65 

Bangert-Drowns, 
C-L  Kulik,  Kulik 

Higher  Education 

.26 

101 

50->60 

C-L  Kulik,  Kulik 

Adult  Education 

.42 

24 

50^66 

Kulik,  C-L  Kulik, 
Shwalb 

^  Average  student  achievement  in  traditional  instruction  mode  improved  using  CAI. 
h  References  in  Table  C-2. 


Redding  and  Fletcher  also  cite  an  examination  by  Niemiec  and  Walberg  of  reviews 
of  computer-assisted  instruction  (CAI)  effectiveness.  The  Niemiec-Walberg  examination 
covered  16  reviews  and  over  SfX)  studies.  Because  of  overlap — ^including  Table  C-1 


* 


Redding,  G.A.,  and  Fletcher,  J.D.,  ‘Technical  and  Administrative  Issues  in  Distributed  Training 
Technology,”  Z^arm'ng  Without  Boundaries,  Defense  Research  Series,  Volume  5,  Plenum  Press,  New 
York,  1994. 


studies— Niemiec  and  Walberg  estimate  the  number  of  “primary  studies  is  close  to  250.” 
Three  of  the  sixteen  studies  were  “traditional,  implicit,  and  nonquantitative.”  Thirteen  were 
quantitative  and  explicit;  eleven  of  these  used  meta-analytic  methodology  and  provided 
numerical  estimates  of  effect  size.  The  mean  and  median  effect  sizes  are  both  0.42,  thus 
placing  the  average  student  using  CAI  at  the  66th  percentile  of  traditional  groups. 

All  three  nonquantitative  reviews  covered  by  Niemiec  and  Walberg  also  concluded 
that  CAI  is  effective.  Among  the  narrative  conclusions,  the  following  comments— one 
fiom  each  review — ^address  the  time-savings  aspect  of  CAI: 

The  evidence  clearly  indicates  that  CAI  will  teach  at  least  as  well  as  live 
teachers  or  other  media,  that  there  will  be  a  savings  in  ti^  to  learn,  tMt 
students  will  respond  favorably  to  CAI  [based  on  15  studies  of  instruction 
at  the  elementary  school,  secondary  school,  and  college  levels]. 

At  the  secondary  school  and  college  levels,  a  conservative  conclusion  is  that 
CAI  is  about  as  effective  as  traditional  instruction  [and]  it  may  also  result  in 
substantial  savings  of  student  time  in  some  cases  [based  on  22  studies  of 
instruction]. 

Achievement  gains  over  other  more  traditional  methods  are  the  norm.  .  .  . 
Retention  is  equal  to  that  obtained  in  traditional  instruction. . . .  Perhaps  the 
most  valuable  finding  in  the  long  run  is  that  many  CAI  students  gain 
mastery  status  in  a  shortened  period  of  time  [based  on  66  studies  of  instruc¬ 
tion  at  the  secondary  school  level]. 

Table  C-2  identifies  the  source  studies  used  by  Redding  and  Fletcher  in  their 
examination  of  the  effectiveness  of  ICW. 
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Johnston  and  Fletcho*  reviewed  33  empirical  studies  that  compared  computer-based 
training  (CBT)  with  conventional  instmction  in  a  military  training  environment.*  Thirteen 
of  the  studies  are  among  the  30  studies  reviewed  by  Qrlansky  and  String  in  Appendix  A. 
The  Fletcher-Johnston  review  involved  using  the  effect-size  methodology  (described  in 
Appendix  B)  to  measure  the  difference  in  mean  test  scores  of  two  groups  in  terms  of 
standard  deviations.  The  standardized  mean  difference,  effect  size  (ES),  is  defined  by 
ES  =  (xg  -  Xc)/S,  where  x  is  the  mean  test  score,  S  is  the  standard  deviation,  and  e  and  c 
denote  experimental  and  control  groups,  respectively. 

In  their  meta-analysis  of  the  33  studies  of  military  training,  Johnson  and  Fletcher 
determined  that  ES  =  0.44.  Thus,  CBT  raised  test  scores  by  0.44  standard  deviations, 
which  means  the  average  student’s  percentile  score  was  raised  from  50  to  67. 

In  a  subset  of  23  of  the  33  studies,  Johnston  and  Fletcher  were  able  to  measure 
average  time  savings  for  the  military  training  courses.  They  found  a  30  pocent  reduction 
in  instmction  time  when  CBT  was  used.  That  result  is  consistent  with  the  30  percent  time 
savings  that  Orlansky  and  String  found,  which  is  not  surprising  since  11  of  the  23 
Johnson-Fletcher  studies  were  considered  by  Orlansky  and  String. 

Table  D-1  identifies  the  33  studies  reviewed  by  Johnson  and  Hetcher. 
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Johnston,  R.,  and  Fletcher,  J.D.,  The  Effectiveness  of  Computer-Based  Instrmtion  in  Military 
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Several  recent  studies  have  examined  the  feasibility,  effectiveness,  and  cost  effec¬ 
tiveness  of  video  teletraining  (VTT):  Defense  Institute  for  Training  Resources  Analysis 
(DITRA)  Study;  Army  Teletraining  Network  (TNET)  Study;  Army  Satellite  Education 
P*rogram  (SEP)  Study;  Army  Command  and  General  Staff  College  (C&GSC)  Study;  Chief 
of  Naval  Education  and  Training  (CNET)  Electronic  Schoolhouse  Network  (CESN)  Study; 
and  Army  Lx>gistics  Management  College  (ALMC)  Study. 

DITRA  Study.  A  project  undertaken  in  the  early  1990’s  assessed  the  feasibility 
of  using  2-year  community  colleges  to  deliver  military  programs  of  instruction  using  the 
Army  TNET  system.  i  The  project  involved  Army  Reserve  Component  Configured 
Courseware  for  three  Military  Occupational  Specialty  (MOS)  courses: 

7 IL 10  Adminisu^tive  Specialist 

76Y10  Unit  Supply  Specialist 

95B 10  Basic  Military  Police 

These  courses  were  reconfigured  fix>m  resident  school  format  for  live  delivery  over  the 
TNET  to  Army  National  Guard  and  Army  Reserve  soldiers  seeking  reclassification  into  the 
indicated  MOSs.  All  students  passed  the  performance  tests  (PTs)  for  the  three  courses,  and 
over  90  percent  passed  the  PTs  on  their  first  attempt 

Army  TNET  Study.  At  about  the  same  time  that  the  above  DITRA  project  was 
underway,  the  Army  used  TNET  to  evaluate  VTT  to  deliver  Reserve  Component  Basic 
Noncommissioned  Officer  Common  Leader  Training  (RC  BNCOC  CLT)  to  Army  Reserve 
and  ARNG  students  at  various  sites  within  the  State  of  Kentucky.  The  goal  of  this  project 
was  to  compare  the  training  and  cost  effectiveness  of  five  training  strategies: 


^  Defense  Institute  for  Training  Resources  Analysis  (DITRA),  Florida  Teletraining  Final  Report,  January 
1994. 


( 1 )  Traditional  instruction  not  reconfigured  for  VTT. 

(2)  Traditional  instruction  reconfigured  for  VTT. 

(3)  One-way  video,  two-way  audio  (IV,  2A)  reconfigured  for  VTT. 

(4)  Two-way  video,  two-way  audio  (2V,  2A)  reconfigured  for  VTT. 

(5)  2V,  2A  reconfigured  VTT  taught  8  hours  a  day  for  7  days. 

The  TRADOC  Analysis  Command  (TRAC)  analysis^  of  training  effectiveness  indicated 
that: 

a.  Students  in  the  IV,  2A  reconfigured  VTT  condition  performed  as  well  as 
students  receiving  traditional  instruction  that  had  been  reconfigured  for  VTT. 
Thus,  VTT  can  be  used  in  lieu  of  traditional  instruction. 

b.  Students  in  the  IV,  2A  reconfigured  VTT  condition  performed  significantly 
better  (.05  level)  than  students  in  the  2V,  2A  group.  One-way  VTT  can  there¬ 
fore  replace  the  more  costly  2V,  2A  delivery  of  teletraining. 

c.  Students  receiving  traditional  reconfigured  instruction  performed  significantly 
better  (.05  level)  than  students  receiving  traditional  instruction  that  had  not 
been  reconfigured  for  VTT.  This  finding  lends  some  support  to  the  expert 
opiiuon  found  in  industry  and  academia  on  the  need  to  enhance  traditional 
classroom  instruction  in  preparation  for  VTT  delivery. 

d.  Students  in  the  2V,  2A  reconfigured  VTT  condition  taught  over  a  period  of 
7  days  performed  as  well  as  students  trained  over  a  4-month  schedule.  This 
finding  indicates  that  there  is  no  need  to  limit  classes  to  4  hours  within  a 
24-hour  period. 

Army  SEP  Study.  This  study^  considered  11  ALMC  courses  delivered  in  three 
forms:  (1)  residential  courses  at  the  ALMC  site  (Fort  Lee);  (2)  on-site  instruction  by 
ALMC  instractors  traveling  to  remote  locations;  and  (3)  VTT.  While  the  three  instructional 
alternatives  were  equally  effective,  the  cost  per  student-week  using  the  satellite-delivered 
VTT  was  25  percent  less  than  the  cost  of  resident  training.  (On-site  delivery  was  less  than 
half  the  cost  of  satellite  delivery,  but  the  study  did  not  examine  the  issue  of  the  availability 
of  instructors  to  travel  and  provide  on-site  instruction.) 


2  Wilson,  Douglas,  “Basic  Noncommissioned  Officer  Course — Kentucky  Reserve  Component  Video 
Teletraining,”  Special  Projects  Branch  Training  Pilots,  Futures  Training  Division,  Training 
Development  and  Analysis  Directorate,  U.S.  Army  HQ  TRADOC,  July  1992, 

3  Redding,  G.A.,  and  Fletcher,  J.D.,  ‘Technical  and  Administrative  Issues  in  Distributed  Training 
Technology,”  Learning  Without  Boundaries,  Defense  Research  Series,  Volume  5,  Plenum  Press,  New 
York,  1994. 


Army  C&GS  College  Study.  This  study^  compared  post-course  scores  of 
four  tests  for  one  group  of  students  who  received  residential  instruction  in  the  classroom 
and  two  groups  of  students  who  received  instruction  by  VTT  in  two  separate,  remote 
classrooms.  All  students  received  the  instruction  at  the  same  time.  Scores  of  remote 
students  were  as  good  as  the  scores  of  resident  students  in  one  test,  and  they  were 
significantly  better  in  the  other  three. 

Navy  CESN  Study.  This  study^  found  that  grades  in  13  courses  were  effec¬ 
tively  identical  for  178  students  who  received  VTT  and  178  other  students  who  received 
resident  instruction,  but  the  VTT  alternative  saved  an  average  of  $386  for  each  of  the  VTT 
students. 

Army  ALMC  Study.  This  study®  considered  a  single  course  delivered  in  three 
modes  of  instruction:  (1)  resident  training;  (2)  on-site  instractors;  and  (3)  VTT  via  satellite 
in  two  forms,  analog  video  and  compressed  digital  video  (CDV).  The  cost  per  student  was 
$2,012  for  resident  training,  $877  for  on-site  instructor  training,  $611  for  analog  satellite, 
and  $419  for  CDV  satellite  transmission,  or  savings  of  56  percent,  70  percent,  and  79  per¬ 
cent,  respectively,  over  resident  training. 
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